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PREFACE 



'this report is ba§ed on the 'work of ten consultants to -the 
National Institute of Education and the 'National Council on Educational, 
Research, The group came together at the request' of the Acting Director 
of .the Institute and th6 Natibnal Council in March 1975, to review edu- ' 
cation research. and development (R&D) funding policies now in effect. . 
and proposed for the near ''future. The review was for the purpose oi, 
evaluating the impact of such policies on the n^ation's education .R&D ; 

• systeifi, witti special reference 'to the regional education laboratories 
and research at\^d development centers established by the government in 

-.l^he 1960s. -r - ^ • .« , " l\ 

This was a. substantial charge, -and. various data-corllectiotu^nd 
analysis aspects- of it merit extensive continuing attention withirf, , 
the Institut'te, ^s we suggest in, our recommendations, ^ore tilne tHan 
the three months available, to the consultants would have pepitted a 
We comprehensive survey of NIE's funding policii^s and podsiljly more 
^accurate predictiotis of their -consequences.. We have attempted to 
relate the pallcy directions we see to, certain data ^available on 
research and development in education, and to other information we / 
gathered. But in the end, we have relied on the judgment, of the 
consultants- to interpret the probable impact of present policies and r 
to make recommendations for illiprovement. 

'^!. A good many people have helped us in the course of' our ■ inquiry . 
Staff at a' number of education R&D organizations, including several 
regional laboratories and research and development centeVs , took time 
to confer with us.dur'in/visits . Sixteen labs and centers generously 
provided extensive aiid 3^ndid written" responses to questions we posed. 
Several dozen knowledgeable people across the country shared with, us 
in writing theJr own analyses of present policies in education R&D 
funding. The Acting Director of NIE and many of his staff m&t Vtth 
us, graciously accommodating our requests for early morning or eveninll 

• hours beyond the call" of duty. The New York State Department -of • 
Education handled the printing, distribution, and receipt^ o^f a^ques- 
"tionnaire quickly and" skillfully, for which we thank our consultant 
colleague- Ewald Nyquist. ' . , . - 

* ^ < ' 

I am pleased to acknowledge als'o the consultants' appreciation 
of the contributions made by our Executive Secretary, Frederick (Frit^) 
- Mulhauaer, and his associate, Maureen Treacy. Without their excellent- 
technical assistance, the, work of^ our group would have been much more 
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difficult. The, support; ol other st^ff in the Disseminatipn and - 
Resources Group at NIE is also acknowledged.. Jh«f consultants aloiie, 
of course^ are- responsible for the 'content 'and style of this report. 

As a group ^ we find ourselves stronger Nig supporters now^than 
when we began. Even so, there are a number of steps NIE -can take 
which in our view will make its funding policies more effective and 
we have tried to speak frankly about thOjse^steps. The report which ■ 
follows represents a ^synthesis of the v^ews of ten consultants. 

; " • From the befeinrrTng it was agreed that any cons,ultant might 
provide an individual statement which would supplement or differ 
^ from the synthesis of the others' views. Sam D. Sieber gave us one 
such paper, on the design requirements of the research and development 
system, which we include as Appendix A. 

I am most grateful to my colleagues for their patient . reviews. of 
drafts of this report. 

We appreciate the opportunity to examine 'the funding policies of 
a the Institute, and the"^ openness of its. leaders, staff, and contractors 
60 our iaquiry. We trust our conclusions and' recommendations will be 
found useful to the Director, the National' Council, and others con- 
cerned w^ improving education -through r^seatch and development. 



Roald F. Campbell 
Principar Consultant 



I CHARGE. TO THE CONSULTANTS 
AND PROCEDURES 

1. 



In a memorandum to the National Council on Educational Research 
•dated April 4, 1975, Acting .NIE Director Emerson Elliott outlined the 
purpose of the .consul tanJte-' work. We were to give our advice to both 
the^Instit^ute and the^Council about "alternative policies Which the 
Institute might adopt for funding education "^L&D act^ities." Within 
this extremely ))road general mandate, we were \to giv\ "special atten- 
tion" to the regional educational laboratoriesXahd research and 
development centers established by the Federal ^yernment in the 
la^t decade. Ejlliott further amplified the charg^: 

^ < » / ' \ 

This advice will be -based ^n- a review of K[IE\f unding policies 

and their effects on various R&D performers ahd will further . 

be based on the consultants' assessment o|: 

(1) the .existing capacity within the nation for high quality 
educational research and development; 

(2) the past relationship of various f-ederal funding policies 
to the maintenance and improvement qf research and develop- 
ment capacity—to meet both current and projected needs; 

d. * ■ i 

(3) the potential effects of alternati^^e NIE funding poUcies 
• . on the maintenance and improvement of such capacity; and 

(4) the relative advani^ages and risks of principal alternative 
funding policies. ' > \ ■ ' 

The memorandum explained part of the moti'^atton for the study by 
reviewing the history of scarce funds and sharply focussed programs 
within the NIE, and the resulting likelihood that some previously- 
supported R&D institutions "will not receive NIE funds equal to past / 
levels of Federal sufport (i.e., ;:heir capacity will be underutilized^ 
by NIE)... The question then, arises whether capacity not now being util- 
-ized by NIE and other fundiig sources under their , cu^rrent priorities is 
of such value to the education community thkt it needs to be. preserved 
through extraordinary efforts."- Mr. Elliott also, stressed to the Council 
the InJ^titute's concern^ to establish and maintain an effective edu^ion 
"R&P system" as called for by the authorizing law, and the InstitutiSf s 
J need for ^dvice on the effects of its policies on the system. He noted 
the particular concerns of the regional laboratories and 'R&D center^, ^ 
which believe that the current Institute plans for Fiscal Year (FY) 1976 



will in some ways damage .their capabilities to- the detriment 1 of the ■ 
nation's. R&D effort. ' -J 



For the consultants, Roald Campbell attempted at an April 4 
meeting with the Council to make clear that the Charge was very broad 
and that the time available, would ^permit judgment'^and advice based 
chiefly on experience and general knowledge, rather th^n extensive 
fresh survey of the state of the nation's education R&D. The Council 
joined the Acting Director in expressing support for the consultants' 
work. ^ 

ToV^y out the charge, the ten consultants determined to use 
the follb\^ng procedures^ 

1. Meet in person with NIE "management , program stitf, and 

seledted project officers; commission ^a review §y staff ^ 
of present NIE policies and their context; solicit in 
writing the views of NIE staff. 

* - 7 

^ 2. Meet with laboratory and R&O center directors in person, 
ir make visits^to selected ins titutions , ai^d gather further 
information from all labs and centers by a questionnaire. 

3. MeeFSln person with representatives of groups which have 

^ interests in education R&D, including teachers, researchers, 
teacher-educators, administrators. Congressional staff, and 
staff of other Federal agencies. 

4. Solicit in writing* the v^Ws of Individuals knowledgeable 
about R&D and related areas. , 

5. Visit selected R&D-performing institutions other than. 
Federally-created laboratories and centers, and others that 
were at one time laboratories and centers, but have ceased 
to have^major Federal supports 

6. Review selected literature on education R&D generally, and 
the Fedjpral role in the field. ^ 

Two weeks after the initial meeting with the. NIE Director and 

Sncil on April 4, the group met with representatives of the regional 
oratories, R&D centers, and the Executive Director of ^ the Council for 
cational Development 'and Research (CEDaR) at Chicago, April 18. At 
that same time, the consultants reviewed the staff paper on the present 
policies of the Institute, and^the context of policy-making. In the 
following month, to begin assessing the state of present capacity for 
^&D, consultants visited' eleven R&D institutions, wrote to over 50 
persons in the field, and arranged a variety of meetings with individ- 
u^s and groups. During the same period, staff prepared memoranda on 
various subj^ects requested, and circulated relevant literature. 
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To gather up-to-date information on the regional laboratorl|ps and 
R&b centers and on the likely impact of NIE policies there, the con- 
sultants drafted a questionnaire and reviewed it with the lab/center 
♦ group at Chicago. After further modification, ten questions were 
posed to sixteen institutions. Confidential responses to one sensitive 
question on^strengths and weaknesses of past relations with the NIE 
were channeled directl:&.to one consultant; the rest were analyzed by 
staff. (The questionnd*re itself, and summaries of "Yesponses to certain 
questions, are in Appendices B and C.) ^ - 

For a third meeting on May ^22-24 , consultants exchanged written 
reports on meetings and visits, and circulated replies to correspondence 
and questionnaires. Based on- extensive discussion of findings, a draft 
of consultants' views was prepared for a fourth meeting at New York, 
June 5-6. Further conclusions and recommendations were considered at 
that time, resulting in additional drafts circulated for review 
and comment ./ 

To gather information Vbout the Institute, the consultants met twice 
with the Acting Director, and with the newly-apnointed Director. In 
addition, consultants were briefed about each of> the Institute programs 
by Associate Directors, and one consultant interv^wed seven project 
officers ffom different program 'areas to understand policies at the 
working level. Consultants^ also asked each program group to rate cur- 
rent work under way ^ laboratories and centers, as to its quality and 
•relevance to the mission' of the unit. ^ ^ . 

Appendix D includes a complete list of place's visited, persons and 
groups met with, and correspondence received. 

> 

The procedures described have given the consultants a three-month 
glimpse at the present state of education R&D, including the NIE and 
certain performers within that system. Literature such as the draft s 
1975 Databook on the status of education R&D in the United States has — 
added a larger perspective . Nevertheless, as stated\in the initial 
meetings with" the Director and the National Council, the discussion, 
conclusions, and rec6mmendations which follow are based on the consult- 
ants' general experience affd contact with Federal agencies, universities, 
research and development prganizatiorts , state education agencies, and 
local school systems, as miuch, as on new data collected. 

Our report is organized into four sections of analysis and dis- 

r . ' . 

W. Paisley and associates. The Status of Educat ional Research and . 
Development in the United States: 1975 Databook (Washington, D.C. : 
National Institute of Education; pre-publication version. May 1975). 
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cussions and' a final section of conclusions and recommendfatloils: 

The chapters grow increasingly focusfeed, moving from con$ide.xation iV 

of the i-ole of research.generally ill education and the resources > 

available for R&D, in ChaptersII and III, to discupsion 6f the 

NIE context and policy in Chf^ters IV land V. Conclusions drawn ' 1 

from the discussion in dll four chapters, and specific x6commendations 

which derive from them, are presented in' Chapfter VI. 





II , THE ROLE OF R&D llJ ^ 
^ improving' EDUCATION 



^ A<broad climate of expectation about research generally, and social 
^ence research particularly, affects policy-making within the NIE.. • - 
Hence we feel it is useful to begin with some attention to more general • 
aspects af the role of research and development in improving educajtion, ' 
In this section^, we note some current voices of pessimism, ofF^r some . 
explanatioTvfof inevitable difficulties facing R&D in human service ^ " 
fields, -^nd en(^.with our own sense of th^ usefulness of the activity 
of disciplined inquiry. • . . . - . 

An observer fn the la^t few years could note a wav^ of criticism 
and uncertainty about research : * 

I * ■ ; • • ' ' ■ \ ■ . 

® A recent monograph sponsored by Ralph Nader attacks the 

objectivity and quality of work by the natioh's most pres- ^ 
/ tigiou© scientific body, the National^ Academy of Sciences. 

V=» A vote 'of the House*of Representatives directs the National 
.Science Foundation to submit 'every proposed grant of fuhds 
for Congressional "review pridr to award". - ^ 

>Wembefs of Congress have expressed strong concern bver the . 
socijil science R&D contracting procedui;es throughout the . ^ 
Department of HEW, and over the support of social studies ; * 
curriculum work and .other social research at the National ^ 
* Science Foundation. - ' . . ^ ^ ' 

' Congress is .even beginning to^questibn some health research, 
as coiltinued biltions invested particularly in cancer studies 
seem to have little immediate result. * 
■* * ■ \ 

And at the same time as these' events at' the Federal level concern- 
ing research generally, one could also note questions raised in th'e^ edu- - 
cation communitr about the -value of educational R&D. Decision-makers ' ! 
are siiid to be pressing for "hard evidence" that new products will der 
liver" the results that policy boards are seeking through accountability 
schemest^ Staff within education, as in other human Services, may be . 
coming' to view the results of research as chief ly'> series of complications 
to their professional lives: recommendations for uprooting the structures 
and personal relationships that once gave security^ within the basic working 
units or organizations or even wholesale condemnations of the present s^J^tem 
and the urging of its rebuilding in some other form. Such "R&l^ result^ 
do not call forth much support from, teachers or other workers in the field. - 

For ourselves, we have noted some characteristics of inquiry in 
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the social or human scienpes and characteristics of. the «cfucation 
system the inquiry is aimed at helping, which perhaps can give per^ 
spective on both the inflated hbpes of the 1960s" and the pessimism"^ 
of the mid-1970s. > ' ^ . • - 

First, in air humarvservice fields,' education included,"" the de^ 
mand for purely service, funiis always exceeds the available resources. 
As a result,. money spent for inquiry and development of ideas will 
always be suspect, as a competitor against supplying "the real, needs" 
of ch|.ldren, the .elderlyV the handicapped, or others. Thus,' we expect 
an inevitable heed^ f or justification,, explanation, and attention to 
the politics of funding. Oversight— by Congrfess or other funding, body— 
of^how research funds will bfe spent, and pressure against siich spending, 
are thus not a temporary condition, but a permanent fact of life. 

Second, the American public education system^ds not controlled 
*frpm a single -point , and is subject to influence from the full • 
spectrum of social force^^r-fads and popular i^hdms, cqurt decisions 
and evolving legal. R0.1osophies, incentives created by shifts in mar- 
ginal funds .available from state or Fgtderal sources; and'the decisions 
of thousands of atomistic actors— clients, providers ^ and. doptrollers v ^ 
,of -education. Furthermore, in a^system so vast^as education, even • 
similar units t^^chopls^di^tricts^, colleges, ^state agenci^^^^ are> differ- 
ent from each other, and^re chaij|ing and developing at varying, rates, 
under varying pressures. The system characteristics of opennesSr, vul- 
nerabilijiy, and complexity have implications for inquiryo,activity : 

.. " knowledge will fnevitaSly be tentative, as the system is 
constantly changing, and even is altered by inquiry into 
it J. 

knowledge will not automatically be universally applickjble, 
, evefi to all like-named pieces of the system ("schools,"^ etc^; 

inquiry will inevitably be seen as a weak tool in a process 
of change and impBoyement, as so many other forces press on 
the system, particularly now the forces of legislation, court 
.action, and tight money.. 

A third limitation on the usefulness of ^inquiry is that even at 
its most successful, the results will not be self-executing. Acto^ 
in the educational system have free will to *some degree, and have 
diverse preferences which they seek to carry out. Thus a research 
finding that certain activities by the teacher can reliably produce 
certain results in a classroom will require a long chain of deliberate 
acti<]^ to produce results — starting with the question of goals 
or ends, ''Does anypne want to attain the results we can fiow reliably ^ 
produce?" 
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Fourth, another consequence of the complexity of^ the process 
being studied in education research is that an enormous range of kinds 
pf inquiry are relevant/ Ve must accept the fact that an^ one study 
or line of research, at ^ny particular time, will yield few %reak- 
^throughs.y Education involves the physiology of eye and brain, the 
psychology of attention, memory, and learning, the sociology of peer,* 
group, and organiz^ional interaction, and so on. , ^Can any qne research^ 
activity be faulted, for not unlocking the whole'' puzzle, as was literally 
possible on some arWas of physical science not so long-ago? 

. • ' . t - f 

These four features of inquiry in education — combine to give us 
restrained expectations for the *role of research and development in 
improving schools. Both Congressional and professional observers 
of educqgtion R&D need to adopt realistic hopes for short-run impacts 
of the* enterprise. Ve all* must moder^^e the impulse* to rip up 
the structure and activity every year^or two if it seems not to 
be 'delivering rapid improvement. Tlie common observation of- the' 
"failure" of social programs of -the 1960s should^jip^ be allowed'to 
\ lead to hasty pressure' for> catching up in ^areas of missing knowledge. 
• A. crash program and a<:companying inflated goals is as unrealistic 
•in, the .-^rea of knowledge-production and utilization as in any 
.of* the servi(# programs of an earlier era against poverty or' other 
problems.'^ - , , 

V For 'the National Institute of Education, we conclude that there * 
is a need for the Institute td speak out more forcefully, to lead ^ 
the deb'ate and tp express its own "goals rather than primarily 
reacting to others. The National Council on Editgational Research, 
it seema to us, has made a good start in talking about the > 
reauthorization af NIE.^ Their minutes show a concern that the 

*NIE not be Expected to have.solyed the problems^ of education, ' bat 
rather to have placed a variety of useful ideas and products ^ 
into channels from which educators can elect to use them. But 
with a new'Director , and a full-strength Council in the near 
future, it will be essential to do a great deal more explaining of the 
grounds on Which the NIE wishes to be viewed — through highly- 
visible statements ^nd more extensive publications and communications 
with the field and x he Congress. 
■ . ■ I ■ •. ' ' ^ ■ 



4r We note familiar rhetoric used recently in a news story ^to describe 

another "attack" and the subsequent disillusionment. "War on Cancer 
« Stirs Political Backlash/' New York Times , May 27, 1975, p. 1. 
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Having urged that expectations be clarified, we reaffirm com- 
pletely the wisdom qf Congress and the ^Executive Branch in esiab-- * 
lishing the Institute, as an important means 

can* education. Thou&h the Institute's life has beei! brief, and it's 
critics .legion, its future potential is fully equal to the Institutes 
of Health or the Science Foundation, and its statyre should be equiva- 
lent.' We find the ui^tapped possibilities so great, even within moderate 
expectations of re'sults both in the Institute's direct support ^of re- 
search and as well in a cobrdinatlftg and planning role for a wide i 
range, of public and private inquiry in education, that we urge long- 
term authorization. Further; though ^^e .recognize the current, 
tight budget, we urge both the^Admlnrstration and the Congress 
not to unduly narrow the future authorization of funds'. The > 
, annual spending ceiling of ^0 mJTllion in the Adminis-tration reauthor- 
'iaation bill is far too low, and should be doubled in the first-. year 
alone. The appropriations process provides a good check on the -actual 
expendi-iure plans; x-je see no need to set in t^ie author! Ang statute 
such a restrictive, i'imit to the potential activity of' the Institute. 

We conclude that there are inevitable limits to the role of in-» 
quiry as a tool of educational change, and that thesfe need to be 

/ recognized, far more than they have been by the Institute, its publics, 
and the Congress. Yet we; also judge that the limits have ha?:dly been^ 
reacheif, and that ^he legal and f inancial*apth6rity tjf the Institute 
should be commens>itate x^^ith the potential and the challenge. In the * 
next section, we turn to consideration of the resources available to 

* carry out disciplined incjuiry into education. 
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^ ' - III THE PRESENT RESOURCES 



iffOR EDUCATION R&D 



V 



> 



Placing the NIE .policies, and their impact in a context has 
b^en a k^y element of our,t^;Qrk..''' Our charge from the first has 
included some attempt to assess the "capacity" for high quality " 
research and development in education in the country, against which 
to test present) and proposed NIE policy directions. .This capacity 
, includes scientific and technical personnel in diverge roles of • . 
researchers, de^^elopers. It includes the institutions i<rtiich 
house these individuals, ^ their varied patterns of structure, 
incentive, and performance. And capacity depends also on the coher- 
^ence-or system-quality of the aggregate, the degree to which the 

' whole is greater than the sum of the parts. We begin with some 
general impressions we, have farmed about the development of ed4cation 
R&D to this point, then discuss in more' detail some concerns we -have 
about the present supply of skilled people in the field, and the forms 

- so far used to arrange them. We colHsclude the section with our assess- 
ment of the relativelysweak integraiipn or inter-relation of the over-, 
all system. ' ' ^ 

- The consultants have been impressed with a number of. features 
Qf' the' expansion of the education R&D effort ±^ this country in the 
last two decades: " n 

'1. tbere has been an impressive history of growth of the 
. re^burces.for education R&D,' iticluding growth in size 
and 6^phist3,i;^ion: / / 

° increa^ in fund,s invested (a trend until recen^ily); 

° wider range of ^talent involved in such work; 

' ° new people added to the resources through training; 

° new institutions formed or dram* into education R&D 
owing to new resources and interest; 

' ' emergence >jDf new technologies and specializations 
such as education research management, sys^tematic 
product development, and evaluation. 

' 2. . There has been a striking continuity of Federal suppdrt for 
education R&D across five very different administrations— 
from the passage^ of the Cooperative .Research Act in 1954 to 
the present, lej^ding from the first funds for research^ to 
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creation of new external institutions in the laboratories ' ' 
and research centers^ and finally , to- qreation of an .entirely 
new Federal aglncy, the -NIE.- 

•There arfpears to Be an increasing volume .of research in- " 
formation and educational produks available year by year— 
though not always readily knox,m/ accessible . or even in 
demand. ^ k 

There, is steadily increasing sophistication of study and 
discourse through the work, of social scientists and others 
/ who look beyond the classroom, survey experts who can pro- 
vide very large samples, analytic techniques to "permit 
large-scale generalizations and evaluation studies of a 

Federal initiatives which c^ be treated 
as rield experiments. 

The, consultants have a?so come to share some less, positive per- ' 

ZHfr/?°"^K'^^ aggregate .of R&D resources, which will'be 

amplified in the rest of this discussion. 



4.^ 



1. 



The numbers of professional and techntcal staf f .rfow available 
for education R&D are markedly .inadequate by a number of 
criteria. 5* • ■ - 



2. The distribution of R&D effort by function and institutional ' 
base, as well as nhe coordination or system-quality, or the " 
whole, leave a great deal to be desired. 

Staffing - . •■ 

^ As the U.S Office of Education's former chief R&D planner wrote 
In Science in 1970, "a primary element in educational R&D policy "fo- 
cusses on how manpower roles and requirements are defined and where 
ttiose kinds of manpower tan be found. Though there appear to be lit- 
erally no up-to-date figures on education R&D personnel, extrapolation 
of past data and guesses based on professional association memberships 
place the total at about ^0,000 people."^ 

Since this figure no doubt includes many' such as university f acuity *' 
whose primary work^is other than R&D, „ there- may be only 7.000 or ^\om 
total person-years devoted to education research, development, and dissem- 
ination. 



Ifl. Gideonse, "Policy Framework .for ^Iducational Re^seairch Science ' " 
4 December 1970, p^ |056. I ' ' 

• 2paisley, The Status of Eduratl nnal Research Chapter A, Sectiofr C. 
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Even if this estimate is low, we are convinced , that me absolute 
numberfe are too small- Such a human resource base is not bearly large 
enough for sustaiuld inquiry to match the complexity, rfengje, and scope 
of the problems in the schools and colleges of the^nation.l Several 
different cViteria suggest this conclusion to us. . i 

'ihfe inadequate size of the/ total effort is^ dramatized by compari- 
sons with research- and development in other^areas. Striking differences 
quickly are apparent if \Je make contrasts with the 10 ,000-person total 
work^orc^^n educational inquiry: . 

° In health, the government's lead agency in health R&D, the 
National .Institutes of Health, itself employs over 4,000 
scientific R&D staff, and reaches many more through its^ 
15,000 contracts and grants to over J,, 200 institutions. 
. f ^ . ^ ^ 

( ' ° In energy, a single one of the Atomic Energy Commission^ 
• ' laboratories typically employs an R&D staff equal to a 
substantial fraction of the entire education R&D group. 



1 



^ The National Science Foundation collects data on industrial 
R&D which show that large firms (with\nore than 10,000 ( 
employees) have 28 R&D scientists and engineers per. 1,000\ ' 
. ' -employeef , and smaller fitms (5,000 to 10,000 employees) 

ha/e 19 per 1,0Q0. Public elementary and secondary ^education 
involves about ?.5 million feachingV^nd administrative 
. employees, for' a ratio of less than k R&D professionals per. 
I, .000 employees*^ ' ^ ^ \ 4 

. Considering %he role of R&I> within Federal education policy, the 
lAanpower seems inadequate also. In the last f^ew years, policy-makers 
have "described the Federal role in two ways. ^ First, the government' 



I sasic. Data Relating to the National Institutes of Health; 1975 (Wash- 

initon, dVc: NIH, February, 19^75),. p. 45. ^ 

2The Energy Research and Development Administration, Public Information 
Office^ gives the estimated employment of scientific and technical staff 
at Argonne National ^Laboratory at 2,500, and at Brookhaven National Lab- 
oratory, 2^400. 

3National Patterns of R&D Resources; Funds and Manpower in the U.S_^ 
l'953-l'97'4 ^(Washijngtion; D.C>; .National Science Foundation, 1974)., p: II 
pn teachers. The Condition of Education (Washington, D.C.: National 
Center for Educational Statistics, 1975), p. 173. ^ ^> 
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leads through seed money or the creation of incentives for atten- 
tion to negl€»^cted groups and issues through categorical, grant pro- 
grams. Second, it wilT strengthen the f oundati6ns- of education 
through support of research, development, and di's§emination of 
improvements in practice. Whether or not this strategy is a wise - 
one^-for instance, in contrast to increased general aid to education- 
its s'^uccess critically depends on the supply of R&D resources. It - 
seems cj.ear that a workforce of the size we see has hardly multi- 
plied' to the extent necessiary to implement the current Federal pol- 
icy. But we are convinced of the long-run importance of inquiry as 
a force for improving education, and if its potential efficacy Is 
undermined in the public and professional mind,^ thdf will be a major 
loss than even* substantially, increased general aid programs could 
not compensate for;. " ' ^ ♦ . ^ 

^Jext, the effort- is too sm^ll when measured against the decen- 
tralized and fragmented character of the system it is intended to 
influence. It is clear that the production of*fesearch, the develop- 
ment o*f new ideas and materials, by some individuals and groups 
separate from the . operating education^ system is not enough. Publishers, 
state^agehcies , itiformation systems, journals, and other "linking ' ' 
agents*' are, of coui:se, parts pf t>he picture. We agree with NIE tha-t 
^ssemination activity will require substantial, increased investment', ' 
but beyond that, probably every large system must\6ave its own R&D 
capacity no matted what dissemination networks e^dst. There are 1,600 ^ 
school;. districts with enrollments over 6,000 pijpils^ and if each were 
to employ a minimal R&D group of five, the nation's R&D workfor^ce would 
need to be doubled, and we will make several* reconpeni^ations urging 
the Institute to begin to see local' and state , education agencies as 
R&D performers, not mere recipients or beneficiaries qf others' worjc*'.^ 



'^Frpm these consiqerati6ns, we do not emerge with a numerical goal 
for R&D 'Personnel towards which to strive.* Ideally that should. r^ult 
not from" "ad hoc cotnparisons such as we have made, but from a program- 
matic analysis of i^at must be done, and how *many people of what sorjts 

* ■ - . . -f' 



It ,has *been argued that the tJrorkforce in education R&D is at its present, 
.size as an accurate reflection o^f demand from the market, *so that 
we are wrong in suggesting an increase contrary to this natural balance. 
We feel the potential of R&D has hardly been given d true test, in view 
of the brief time it has had substantial support, the primitive nature 
of the dissemination system, and the hitherto weak results from much 
research. .It seems faii^ to call for some further subsidy of R&D under 
stable conditions of production and field dissemination to at least 
gather further data on th^ issue /)f demand. \ ^ • ^ 
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are needed to do it.. " We hope the enlarged capacity for monitoring 
and artalyzing the R&D system whiph we wilf recommend f or* NIE cQuld 
be^in" to provide the data on whic^ such a plMri ;could be built. Such 
data include better figut^s on the present location and types of 
R&D. personnel and analysis of policy issues such ^s how best to 
strengthen staff, for the fui;ictions needing to be performed in the 
preseiitly-underserved placesv isuch as school districts • 

' ■ ** ' \ ' ' 

in the absence of such a plan, We favor the relatively simple - 

approach of providing incentives .to kfe^tract the ver^ best quality 

•of people into the research and development field, both new entrants 

and those already established in othetv fields. This approach does 

not set ^ number o^ professionals to be reached at a certain date. 

It simply provides a mechanism for the Wstem to" ^:^pand and then 

throttles down the incentives at some PQint whe^e intuition'^nd 

collective judgment indicate the system Vii^ getting too large. We 

believe this yjas partly the history* of \t^ia growth of the natural 

sc^.ence^ in this country and has wqprked Vi^^asonably well. ' 

•The nature of the incentives is ^irly clear, including the 
! following elements: ' -> ^ - r^^^ " . 

° Attracting recenj: graduates into^ the; field because of 
the availability of jobs; this is\^s^eciaily possible 
i^ th^ next decade pw'iijg to the ovef^^upply of PhD- trained 
individuals, A Substantially incre^sqid R&I) effort in *• D 
education thus can taHe advantage of^a £;dJLden opportunity 
to attract sopie of ttfe best C>f the c%r'ent crop of sttidents. 



- • Attracting new students by the traditional fellowship 

incentives — though we favor parfial se^f-s.upport by each ' ^ 
student; despite the current Federal policies againat j " 

fellowships, they are a logical outgrowth of our argument 
about the need for growth of staff. 

^ Attracting already-trained persons who ai-ei now In other 
fields, but who could make . contributions^n education FT&D. 
\ This could be done thijough Tnid-career edup^tion, change-* 
of-field grants, or other incentives that^^upport and 4 
easQ the transition. ^^ 

Particularly in view of our emphasis in later Bp ct ions on the* need 
for high-quality wark,' we find thfe present st^ff si^^ for ed^cati^ R&D 
simply inadequate, and we wanted at the outr^sjet of oiii^ assessment to tall 
this fact to wide attention. ^ ^ 

Our judgment as to ?he aggregate shortage of R&D professionals does ^ 
„not mean we then find all the present people to be of acceptable quality,^ 
•all present institutions worthy of Support, or all R&D ^f^nctions in their 
proper balance and relationship. We do*,' dnfact^ have some more refined 7 



perceptions as to^bolih the functions and the institutions, and 
about the system's inter-relatedness, ' 

Functions f , 

• • • ' . ' A 

It is difficult to know exactly what functions are presently N 
performed in what proportions by the R&D workforce. Tlie 1975 
Di ^book (referenced earlier) gives, without explaining the 
method used'', an estimate from one survey, which is shown in fhe 
first column of the table below. We also asked '^ach of the regional 
laboratories and R&D centers to describe each 'cu;ftent activity, to 
categorize it as to^ its major purpose , and to give the number of 
professional staff assigned to each project. A tabulatiqn of those 
•reports is* shown in' the second and third columns below. ^ ' ^ 

c ■ " • ^ ' ■ , . - 

Distribution of the R&D Workforce 
Recording to Various Sources 



^ ' ' 1975 Databook Regional Lab Center 

Type oT"work Report , ' Survey Survey V 

~. ^ ' ' — ^ 

Research * ^ • 33% 10% * • 2^ \ 

Development 50% ^ , 61% - 53% 

Diffusion - dissemination 17% " - . \ 3%- ' '6% 



Despite gross definition problems in- these figutfes, the rank 
orderl^ng may^be taken for some indication of imbalanc^.. One set of 
cr»Lteria of balance are proposed in Sieber's papier in the appendix. 
If only in. political terms, the present allocations are undefiirable^ 
and in terms of testing the true demand, for R&D. also, the inattention 
to dissemination makes f ar^ a weak experiment. Thus, while we hope - 
the research sector can gfow, we hope the dissemination sector can 
grow even more and that the development staff can be betj^^^distributed 



The totals In the second and th'ird columns do not add to 100% be- 
cause Wte omitted staff on projects categorized in several domains, 
and some staff were in work outside R&D' completely, such as training. 
The figures are obviously weak in another respect, since within a 
"development project" there may be researchers, de^velopers, dissem- 
inators . 
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On the research siBe, especially in the areas of mor^ 'basic studies, 
the kind of people who can do good research ai;e relatively rare. 
Therefore, we are not talking about a large increase in this "cate- 
gory. But in addition, the nature 9 f development and dissemination 
activities area, such that they require group and team work, specialisa- 
tion of functions and divisit)n of labor, and they must take place in 
many diverse locations. ^ These characteristics set up manpower re- , ^ 
quirements of much grecKer magnitudes than in basic research certainly, 
and even than in ifiost^ applied research. And especiaLly if systematic 
program analysis and' development^ is increasingly to take place within* 
schpol systems, as we feel it must:, and as state and school-district 
leaders tell us they agree, tha^ set of R&D staff will need to be ex- 
panded. ^ ^ 

Institutions 

Turning, then, to the organizat^Tons^riOiit-available -as /settings 
for education research and development, we find' an adequate^ rang« 
tn existence. Clearly we 'do need institutional structures beyond \ 
the un,iversities ai?id service-p^ovi^ers themselves. And we do need 
to "make ^ a- market" so that* good ^research people can find^areers 
*^other than in the uniAJ^ersity . different structures attract-people 
of different motivation, also; To cite energy R&D again, tho^e ^ 
national laboratories are able to recruit and ^hold outstanding 
engineers and scientists, persons who could command much higher 
salaries in industry, because suth indi^fiduals wished to work 
under the moire settled conditions of a government- funded labora- 
tory, gn the other hand, there are excellent professional staff 
who do thrive^ on organizational change in any field. 

.Our point is that there are dif f erent ,kind^ j^f institutions^ 
which seem to us necessary to organize the diverse kinds of talent 
edircation must call upon in R&D. These ' include the for-profit 
<af^research, data-handling, testing, and publishing firms; those non- 
profits which also delight' in acting in an entrepreneurial fashion;- 
th6 regional laboratories; university-based , individuals and groups; 
and the state, intertnediate, and loc^^l education agencies. These 
seem to makte up a balanced institutional complex, or at, least we do 
not immediately conceive of a type of R&D setting omitted, ai\d essen- 
tial, except possibly for intramural ^research at the NIE itself. We . 
will make>some comments about the capacities and present utdlization, 
and some ways capacities might better be used, for each in turn. 

Independent firms . The entrepreneurs, either for .profit' or not, 
represent a quick-respons^ capability, and an almost incredible flex- 
Ijbility of structure an/ persoiinel depending upon the njarket. As, 
cwo of our group reporte4^"f ter a visit to one such place: 

The interesting phenomenon is the difference in expectations - ' 
. from what the laboratories and R&D centers "have. (The research 
x:enter we visited) is Treally a business -and infused with the 
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ethic of biisiness. It is a supremely confident organi- 
zation^ which feels that unless the dice are loaded it 
can compete effectively with other comparable organiza- 
tions or labs and centers. They maintain they have the 
ability to attach and detach staff easily as problems 
arise on which government or other parties want help. 
They make extensive ubb of consultants to get the flexi- 
bility and the' expertise they know they need. They appar- 
ently have not only tF^ ability but an/ enthusiasm for- 
managing in a situation of uncertainty. They, exist on a 
year-by-year\ contract-by-contract basis, and want it that 
way. They bdlieve it keeps them on th^ir toes and keeps 
them lean,^ anjd as a result t^y welcome annually facing 
'the test of specific contract renewals. Naturally, they 
tend fco recruit people who are not devoted to tenure-type 
situations, ' l 



It seems unlikely that any other types of organization will ever 
have the ability to produce research-based analyses with the' speed re- 
quired by emerging policy debates, to ,^fg±ti national 'studies withih 0> 
months of the award of funds, or to do'any other tas-ks that are not 
rewarded in the university or where the university time-perspective 
is too long. There are a number of firstyrate such entrepreneur? in 
the^ social science ^field; and they should not be excluded from t^e 
government's R&D effort; but .should Jje recognized for their unique . 
capacities , ' 

Universities . It. seeps-to us that universities will continue to ^ 
include many^ of the first-rate scientists and scholars who are work- 
,ing to extend the basic knowledge upon which education ultimately rests, 
Indeed, because the range of relevant scholarship is so immense, its 
organization for productive^work on education is most difficult. 'Oar 
most basic consideration aftbr looking at the capacity of universities 
to contribute to education R&D is that only a few models of such organ- 
ization 'are evident — the R&Q center and the single-investigator,* and 

We note the interesting case of .g^^cational Tes^fe^ing Service, a non- 



profit agency where an educ^|:l^^lE^roduct or 3'^ylce'.;^( tests) has also 
generated substantial basic Snd ap'^p!!?!^ RM) ip the firm,' Until repently, 
perhaps,^ educational materials publishers spent- little on R&D, no doubt 
as their market was not basea^^jon factors that> could be improved by education 
research. This may be changing, as public systems and their suppliers 
feel the effects of pressure for accountability. We also note the 
occasional independent research allowed employees of RAND Corporation. 
Glearly^ the location of R&D, and its diverse sponsorship, . is a complexx ' 
que"Btion which must: continually he addressed by NIE data-gathering and 
analysis. ' ' . ^ ' 
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'that theV'needles^iy limit our ability to use that capacity. 

' .The need for involvement in education, research of scholars in ' 
basic disciplines, and' the need to move beyond the isolated work of 
individial researchers, led th^e^U.S.Off ice o^^^^^-^^lVthr 96^^^- 
the university-based, research and development centers in the 1960s. 
3ne Tf our correspondents, closely familiar with the oritins of the 
Sntersrsuggests that whatever tl^e mechanism thc^ is \contlnuxng 
neeHo; support of basic social Science in relatidi to e^cation: 

From the .very start.' the program for R&D In education, 
took these t^cts (of little other support for b^sic social • 
» science) into account and tried to get good social sci- 
entists to work on education programs... The situation is 
still the same. In fact, tlie Federal government seems less^ 
likely than ever to :set.up a program in support of basic 
. social science, research, for obvious political reasons. 
And I take it that other funding sources are not lllcei,y 
to take up the baton. , - . ^ 

If that is the case, then educatiorial research is still 
in the fix it was ten years ago; it has to smuggle in 
basic research in the social sciences while it pretends 
that its tfrogram is dfesigned to concentrate on educa- 
tional matters. Nor does there seem to have b"";^ 
notable increase in such basic research by educational 

■ reseachers. We cannot, therefore, stop worrying about 
, education's external intellectual bloodstream. . . 

■ ' And in addition to the continuing need for sustained work oVi 
fundamental problems from a variety of^ disciplines we -"J^ note that 
t^e organization of the work should in"clude a '^""^^"f 

world of practice. .In this way. the advan^e^ students who must Weas 
Inglyfin'd. jobs outside the -^-r^^^^^ j^ .^.^.^^ - 
relate .to policy and practice, and in additioji. the university 
stay aware of the needs and realities of the system. ^ 

However, university researchers are in maJy ^a.ys unsuited to t^e 
demands of tight timelinL and pre-specified oljjectivesno be attained. 
wmS often accompany Federal R&D funds.2 problems in education are in 

lADoendix C lists thk present R&D centers, th^ parent universities.^ 
^d other data. ion sta^f and budget. The history of their establishment 
is well summarized in the 1975 Databook . Chapter 4. Section C.^, 

2jai„es Coleman's essay on "Policy Research in the Social Sciences" 
Srristown. New Jersey: General Learning Press, l^^^) is an excellent 
discussio. of how universities/ shotild not be expected to^erform certain 
kinds of^tudies intended to quickly irttluence policy. The cancer 
example cited earlier is another, instance of pressure for solutions, seen 
as inappropriate by basic researchers. 
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,ny cases millennia oW, and a university group pr any, other' for that » 
flatter, can not be expected to find a breakthrou^ where many^have already 
strufigied and made J.ft.tle headway, particularly if we expect /the diverse 
apprfoaches of a number of disciplines to be iivtegrated. < * 9 ' / 

■ ^ o' ' V • " ' ■ ■ ■ / 

The nine present R&D centers, have a rdnge^of wdrk under vf'ay, 
from heavy concentration on development and marketing of products^ 
to more detached work to understand educational activities, they 
vary, also, in^Jthe degree of emphasis on a common mission within a 
center. The quality and usefulness of their work varies, as judged 
by^ educators in the field and scholarly colleagues, and it seems 
fair to say that owing to the diversity of "the work, it woidld be 
hard^to judge "centers" accordirlg* to a common criterion siich as qon- 
tribution to knowledge or help tc the field. 

- Organxzmg university talent for large efforts in education . 
R&D inevitably confronts issues such as: 

* ° , the balance between autonomy tn staffing; and ties, to 

the departments; * , ^ 



independence of action and research vs. closer ties to 
the governiJFent; * ^ , / . 



° term of funding. ^ ' " v r ^ 

( We^ think that past pressures for rapid develppmenfc">nd evidence *^ A 
of "impact" have probably forced many centers to neglect basic research and tcr 
assemble people who could wbrk for immediate cesults under/ pressures considera- 
bly diff^ent from £hose.in typical academic life. The r;Bsultiig la^ck 
of consistent interchange with scholars in the disciplined seemis a - I : M 
decided loss. Again, ^e government's need for deitions^rabie produc|:$ 
has probably \e0 to clo§^ ti'es to funding agencies than ur^iversities 
typrically would wish, andi;h^ terms pf funding tnay become porter,* w;Lth 
repeated competitions in ordfer to^ maintain pressurfe and affe<^antabillty 
These developments seem to be bafriers to taking fuli advantage of university 
scholars, and seem almost designed to t;urn segments of ^the university into 
independent and Tower-status agencies. V 

If the R&D center is one model presently in use, the sing^e^- 
scholar or small team is the other. Many \ of these were repreeeni-ed in the 
^ield- Initiated Studied competitions held by the NIE two and tjiree years 
ago (in which 82% of the funds went to .colleges and. universitieV) • We 
agree that diversity of arrangements is necessary, and we do sfee a^ 
place for support of reisearchers who do wot wish to participatj|e in large ' 
organizations.* Incjeed, the superior researchers in any fielj|^,who "can 
obtain funds. Easily see tio need to hay-e th^ise dollars siphonfj^d off to 
"support "overhead,"- see no reaspn to join with others of lelj^er. rank, 
and a& a result^ are not often fgwnd ip "centers." f 
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^ , However, our general view of university 'capacity , and its 
pr/esent use, can be summarized thus: 

1. The need to support basic work in social science is ^ 
as great today as when the R&D centers were first. 
J established.^ ' ^ , ' 

2. A very wide range pf university talent in numerous de- 
p^tments and professional schools, can be related to " 
thk national R&© effort in education^ ' ^ 

3. While the university effort in education R&D needs ^ 
for^yari-ous purposes to be related- to the world 'of- ■ • , . 

-practice, the work should not be judged by.lts » immedi- 
a'te contributions to solving specific problems in ^* 
^ ' the fi^eld. / ^^ ' . 

4. Arrangements to capitalize q(n the talents at universities 
must take a great many forms, no t^ simply centers or iso- 
lated professors, and must take into account the need 
for relative independemze and long-tejm support^ for 
maximum effective use of that talent. , ' 

5. R&D in education must be organized at universities' in such. 

a way as to maintain the links with departments in the ' 
disciplines beyond education which are its intellectual 
^underpinnings. 

This^l ast two points are most important to us. We feel it is part- 
icularly essential to imagine ways of relating to universities that^ • 
draw on the best people there, r^nher than relying -exclusively on a separate^ 
entity which establishes its own staff on soft funds, potentially unre- 
lated '^to the academic and intellectual life of the rest of the university. 
The arrangetents used must be apt for engaging the present sc^holars, 
where they have contributions to make, not establishing a second-class 
set of citizens, impermanent and • isolated. 



Sam Sieber of bqr group has argued in his book Reforming the University ; 
The Role of the Social Research Center (New York: Praeger Publishers, 
1972) that the "integrative functions" of university research centers ^ 
are severely hampered if such agencies are not able to operate as secure 
and stable entities. ^ He sees this integration takingplace as the cen^ 
ters span boundaries between usually distinct areas: university and 
society; research and service; student, and teacher (through research 
apprenticeships); education and basic disciplines; intellectual work 
and management. 
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In addition ttx persons in regular departments and the schools 
of education, there are other resources in universities not much tapped.^^ 
at present. These include: 

i: o Schools of business, which have a growing interest in 

f public management^ 

? • 

^ o Policy-science departments or institutes, where people 

are coming together around general issues in the design, 
implementation, and' evaluation of public programs; 

o Schools of public administration, ^here focus is shift- 
ing' towards policyymaking, as well as its execution. 

We do not have a clear formula to suggjest that will in all cases - 
hook the right parts of universities together for various education R&D 
purposes. We do ha^e a sense that more options should be explored^ as 
the present capacity is under-used and its^procurement bound essentially 
by two limited concepts — the center and the individual or small project. 
We have in mind a least two other modes'. 

1. An up-do-date survey should be done of university faculty^ 
and the departments,.' schools, and institutions in which they 
work to determine their- potential fox an interest in educational 
R&D. Based on a determination of high potential,, the precise 
form of the organization could well be left to the university 

or a major segment of it, with considerable!' discretion left to 
the grantee to put together a combination of people in whatever 
form and manner is appropriate to it. If this turns ^out to be 
through an establishecT R&D cetiter, so be it. If the mechanism 
is an internal university foundation, so be it. 

2. When the government defines certain work needing to be done, 
and finds scattered university resources available Ithat may 
not naturally come together in a joint effort, procurement 
rules must be^ interpreted to allow the government to bring 
the parties together, broker agreements, and emerge with a 
total resource greater than the parts alone. 

We have been told several times of the success of the National Science 
•Foundation in drawing world-reno^ed university figures in Boston and 
Cambridge into science curriculum development. The meSMfcwas a loose, 
time-limited entity called Educational Services Incorpo^ffed (ESI), whidh 
had a few central services, but chief ly organized the part-time efforts 
,of many others from the university community. While we feel this example 
is in many ways unique, it ddes underscore k general point, that education 
R&D in the present era need not hold^to one or two ways of organizing 
university resources. We find special merit in the argument drawn from 
this example, that we need not be bound to permanent institutions, 
in universitites or anywhere, that live on after their missions are 
Completed. * ^ 
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When we return to specific NIE policies, we v/lll discuss further 
our notions of reducinaj, the present number of centers, which would 
' give room for creating^ Variety of new arrangements thab suit the NIE 
program areas and are tlstter^ adapted to the special circumstances of 
universities- - 

Laboratories , Igve third set of institutions whose capacity we 
must address is the regionjal educational J.aboratories , established by 
the U.S. Office of Education ^porl passage of new legislative authority 
in a 1965 amendment to the ddgfeerative Research Act. As the Background 
Report on the "Labs and Cente^s" ^ outlines, twenty were created ori- 
ginally, dispersed throughout/ the country as the name implies. As 
appro^priations failed to iHt^^^se , nine were closed by USOE in 1970, and four more 
have ceased to receive Federal funds siace then. Even with these reduc- 
tions from the original' numbers , by the time of our survey of the re- ^ 
maining seven laboratories in April 1975, they^^iflp-i^ed almost 700 / . 

full-'time equivalenrt professional staff, or close to'^en percent of / 
our estimate for the nation's Entire R&D workforce. Frarther, their / 
annual budgets in EY'^l975 totalled over ^$20 million, $1^ million of ./ 
which came from NIE,^ ' . / 

Our general feeling, aft^r considerable listeningj^&^lea5iers of < 
' the laboratories both together and individually, examininf' lengthy: - * * 
questionnaire returns, talking with past and present Federal govern- .T 
ment staff, and having the benefit of the written views of several ^ 
dozen observers of e(lug,ation R&D, is that the concept was distinctive ^. 
and . important wheti fii^t broached, and remains sound. However, we ^ / 
. are wide of the mark in many ways in the present implementation of Xy. 

By "the concept," we mean at the most basic level, the idea of 
establishing long-,term R&D .institutions distinct from both the univer- 
sity and the operating education system, and not in the market to do 
odd jobs of research or service, but to caUfry on substantial work on 
complex problems. The task, force which originally framed the idea 
had in mind the creation«^f a small number of high-quality national 
laboratories comparable to those of the Atomic Energy Commission, and 
perhaps with .other features similar to some' of the clinical facilities 
of the National Institutes of Health. 

^Prepared by NIE staff for the National Council on Educational Research 
for its March 5, 1975 meeting. r ^ 

Appendix. C gives further data on laboratory staff and budgets. 
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, • How haa the concept weathered a decade of efforts to implement 
it? One former official of the U.S. Office of Education who was close- 
ly Involved during the fcreation of the laboratory legislation Offers 
thlj further personal recollection -of what the laboratories were to 
be: 

Th^ laboratories would cafyitalize on (the potential of 
great advances in basic knowledge) by devoting atten- 
tion to basic research but would devote prime, attention 
to "development and -dissBmination of educational inno- 
vations." The laboratories would have strong links to ^ 
state departments of education, to school systems (par- 
ticularly for teacher training and field* testing) , to 
universities and ir^dustry.^ Every laboratory would have 
one or mpre experimental schools %ore or less under its 
o^^ Jurisdiction." Together, the laboratories would 
form a nationwide network to. test the feasibility of 
new methods. They would also -estaUlish effective chan- 
nels of communication among themselves, collaborating 
easily and continuously. Teacher training was to b^ 
an integral and major part or the lab program, with 
new models for the Education of teachers emerging in 
the process. , 

f - 

Despite these specific early hopes, the laboratories in fact cam^ 
to enjoy a relationship with the U.S. Office of Education ih which they 
proposed work they wished to carry out, and the Office negotiated and 
agreed. The Federal R&D officials lacked sufficient time or mandate to 
develop long-range priorities against which to judge laboratory work • 
plans, and the initiative thus rested effectively with the labs them- 
selves, in competition only with each other for shares of^ a specific 
segment of the USOE budget. In this way, laboratories were able al- 
most to set their own agenda, and to come up with their o,wn best sen^e ^ 
of what functions to perform, for whose benefits, with what'tools, 
Each institution worked under a single contract. 

^ w 
^ In 1972, the USOE decided tlJat this relationship should be ^ 
changed so that the government would henceforth be purchasing^ specific . 
programs ofvwork, under separate contracts for each program. A coXract 
price and term would be negotiated at tl^je outset, with products andl a 
schedule for deliverii^g them. The NIE inherited this incipient policy, 
and concluded the latter part of a review process begim at USOE to 
decide what specific work should be supported and for how long. 

1 . ' * ' I 

Samuel Halperin, "Title IV of the Elementary and Secondary Education 
Act of 1965: Executive Aspirations and Legislative History." (Pr4-' 
pared for presentation at the 1971 anAual meeting of the American 
Educational Research Association.) 
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Thus, the laboratories were ^seen at first as a unique institutional 
capacity, able to pian and set their own direction subject aJ^y to constraints 
of Imdget and. occasional internal and external reviews'. As funds stabilized, 
as judgments had to be made about terminating Federal support at some 
laboratories, and as specific l?rograms of work have become the focus, 
questiolis about the cost-effectiveness of this form of R&D and the impact 
of its products have been raised more and more. In this second era of 
relationships, the laboratories^ seem to have been viewed as organizations 
like any other, which had agreed to do a job, and should be judged on tiie 
work being done. In a sense, the government could argue that its R&D 
management approach has fiiaturedi from a time when simply establishing 
some ne^^7 institutions was in itself the major challenge, requiring 
considerable discretion left to the field to carry' out the nov,el kinds 
of work everyone had^ in mind, to the recent years when managers fe:f.t ^ 
more confident in. the assumptions that the system could taTce care of it- 
self, that adequate institutional capacity existed, and th^it the focus 
for federal t funds could shift to major R&D priorities determined after 
wide consuitatipn wiCh both practitioners and researchers. 

As a legacy of the early years of laissez-faire , the seven current 
laboratories differ enormously from each other, and the specific work 
they were contracted to do after the 1972 review reflects this diversity, 
as it was, largely based on what had gone befdre. So we find laboratories 
at present varied in their sense of what schools and students need, 
in their internal governance and policy-making, in their ties to their 
religion, in the degree of emphasis on service to the local schools 
nearby', in the balance of functions performed such as research, development, 
dissemination or evaluation, and in the degree, of programmatic coherence 
and mission emphasis. Also, they differ in their current degree of 
dependence on the NIE. ^ 

Thus the laboratories are a patchwork of capacities ' and interests, 
and though we could hardly evaluate each laboratory against a set of 
criteria, our impressions (and those of our correspondents) were of 
very diverse quality with respect to any specific capacity such as - 
product development or evaluation. In this connection, we recorded from 
our discussions in the iield some serious issues and criticisms concerning 
the labs' present capacity. Among these were: 

o Major resources devoted to curriculum designs and materials 
of instruction, without adequate study of the schools' need 
for them or their appropriateness to real problems. 

S ' " ■ 

♦ o Little sophistication in mark-eting and sales, and unfortunate 

relations with commercial publishers, ovie of whom described 

as follows relationships with a laboratory: "we were summoned 

like errand boys, assumed to^have no educational views, 

editorial abilities or traditions, but lots of money — and^ 

told to peddle these brilliant programs. 
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This style has probably changed in recent years^, but the/, 
memory lingers on. Unless a better relationship is est- 
ablished in which the publishers are asked to the dinner 
as^well as the showv I doubt if the best "disseminatioa'' 
instrument available will be made effective.*' 

Actual products themselves often not demonstrably better 
than th^ range of commercially-developed materials, and 
frequently ^ore expensive. 

With certaifn outstanding exceptions, laboratories per-, 
ceived by^mporUbnt practitioner groups (superintendents, 
teaph^x^ as distant, unhelpful, essentially similar to 
ot|her consulting or contract-seeking en treprenetfrs. 

io strongly wedded to conceptions of educational change 
through hardware\nd software; not adequately interested ' 
in system-change and change from outside the establishment 
^(even granting the limits of R&D in th^se areas). 

Without close ties to universities, seep as not equipped 
to exert academic forms of quality control, nor closely 
regulated" by boards or reviewers, and the result is a very 
wide range of quality of staffs, coherence 6f missions, 
tautriess of management, and utility of products* 

Some of the most devastating comments about the laboratories 
came to us from teacher and administ;rator groups, which is particul- 
^fljfly troulu^ling in view of the substantial funds already spent «ln 
'developing products aimed at helping these two groups. 

Nevertheless, despite the criticisms we have weighed, for us the 
only question about the basic concept of the laboratories is how to 
make it work well, not whether the laboratories should exist. The 
need for established, long-t^rm, R&D institutions still impresses us. 
And we believe that a good many of the complaints about the quality 
or orientation of the remaining labs must be laid directly at the door 
of the i*ederal government. Tne Federal ^government created the institu- 
tions to meet certain needs, and if the needs Are not being met as well 
as they might, it-seems plausible to us^ to examine the Federal govern- 
ment's leadership and management, rather than to. reject the concept of 
the laboratory structure. 

. . I ■ ' " 

However, saying we believe tljfe need still exists, and that the ^ 
original concept of distinct ^organizations .to meet it is sound, we 
must hasten to add that at present the concept i& out of control 
and. being implemented in unsatisfactory ways,** and that there are still, 
too many Institutions (given the shortage of quality R&D personnel) 
to insure the uniformly high-quality of work orig^in^ly hoped for. 

-> 

■ .. ^ 
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Furthermore. because the federal government 's .echication R&D expenditures 
have pot.rlsen as^ many hoped, and because it is important to invest in a , 
variety ©f approaches to education R&D and to consider the proper pro- 
portion of funds that should go to any particular set- of -performers 
we come to the conclusion that the laboratory program must be limited 
to allow reasonable support for ' the activities of other important agencies. 

^ ■ 

' In thus reconsidering the Concept of the laboratory and how to 
move deliberately from the present situatipn^tp -the future v^we" have 
in mind certain principles which we think should characterize Che 
capacity of the "new" national laboratories which we expect, to emerge 
- in the next few years as one key par*t\of the R&D system. 

First, 'there should be a small number of very high quality in- 
■ stituliiona, perhaps no more that), a half dozen. Though we later pre- _ 
sent some budget considerations, these are not controlling. We think 
there are only so m^ny top-flight staff available for^such places, 
whioh sets a limit on. the number th'at should be supported, and we also 
believe that other institutional arrangements deserve attention beyon^ 
the one we are considering here. 

Second, «each institution should center on a #Bsion, closely - ; 
related to a priority of the major sponsoring agency, the National 
Institute of Education. The work of the/ organization would be to ; 
take ideas from their inception through Vtheir development, refinement, 
and testing where 'appropriate , to' dissemination. The substance to be 
. worked on could "range from^ classroom instruction to systems of school 
finance. ' ^ «i ' 

Third, the organizations' funding must meet several standards. 
It must be' stable (three to five years), it must ^chiefly ccme from oi»e 
source (the NIE) , it must be clearly tied to. specific work agreed upon 
in advance! and it must be of a magnitude of $3 to $A million per year 
at least. We believe that quality work on complex problems requires a 
clear sense of mission, relative freedom from the distraction of 
searching for funds,' and the assbmbllng- of a sizable team of peo^e 
that can work together over time. 

Fourth, the organizafien must be protected from demands to give * 
undue services to'^ocal and state agencies unrelated to- the major R&D 
mission. Any tie; such-as to a state legislative comniittee or local 
school baard, should be in the ^service of working out an idea of wide 
applicability, as .a way of improving the national R&D product. Thus 
• individual organizations might or might not be working with agencies 
near them. . Testing, for It^stance, could, be going on in a distant state. 
We believe such ties to the field at all stages qf inquiry are essential 
' for the laboratories' purposes, but we are cautioning against seeing • 
a small number of national laboratories as places where practitioners i 

/ . 
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might go and expect advice on education pfoblems generally. - ■ 

. o , * - « 

Fifth, it' should be clear that the unusual guarantee of contin- 
ued funding will bring with It a need for an unusual degree of. monit- 
oring and review of the work. Much more than at present, an -organiza- 
tion which wishes the kind of stability ;we have outlined here must 
expect close review of policy and management by the sponsoring Federal " 
agency, NIE. The special- relationship of insulation from repeated 
competitions for short term funding makes other forms of quality control- oti the 
glibseantive'work especially important. The relationship of policy boards to 
their institutions mass^ need reexamination and change as the Federal role ■ 
and mission become more significant in the renewed laboratories. 

- Sixth, the pursuit of other funds b^he organization should be ' 
subject to review and perhaps limitation, in- order to insure the 
focus on the major goals of the chief sponsoring agency, I^IE. Addi- • 
-tlcJnal funds which .supporf activities closely related to the main 
mission would-be welcome, but active search for other types of ^unds ' 
would raise questions about the commitment , to the major missi'on.^ ~ 

Seventh, the recleslgne'd' laboratories would no longer be desig-' 
-nated regional," though we db feel they should be located in dif- 
ferent part^s'Pf the country. So long as they expect substantial - ' 
Federal support in an era of verjf tight Federal R&D dollars, - they 
muSt lie working on parts of the national R&D agenda, as set through 
a national process. The institutions would, of course, cohtribute 
to that planning process, through their knowledge of local issues, 
'but. they would not be expected to independently address local problems 
through R&D. . 

Eighth, lastly; we are doncerned that as such strong and unique 
institutions mature further, they maintain a prime commitment to effect- 
ing change and improvement in schools. Their closejiess to the govern- 
ment, and their somewhat detached R&D wofk may reduce the vividness of 
the daily life of students and teachers in .classrooms. We heard too 
much criti9dsm on this score concerning the present laboratories and , 
centers, and we^note^the unfulfilled aspects of Halperin's list of . 
original hopes-'-close ties to teacher training, maintenance of experi- 
mental, schools , and constant attention to have R&D impact pressing 
needs of students and teachers. No amount of sophistication In the 
R&D work can compensate for irrelevance to the world of educational 

We. dp not rule out such funds, and we are aware of the Commission on 
Government Procurement view that institutions such ^siae are proposing 
should not receive all funds fr*m a single agency. ( Report of -tha " • 
Commission, Volume 2, p. 18; Washington', D.C.: 18^72). Nevertheless, 
we feel strongly about the need for consistency, and goal orientation 
of funds, and outside, review may be needed fo main^in that. 
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practice, and the renexved laboratories should not forget this* Even 
such small details as the lavishness of facilities and the size^of 
salaries may set a tone of separateness that can be harmful. 

The difference should be clear between these aspects of our con- 
. cept of the national laboratories of the. future, and either the Earlier 
U.S. Office of Education notion of supporting independent institutions 
which set their own agenda, or the current NIE concept of purchasing 
discrete products from an undifferentiated set o? institutions, 

' \ ^ ^ 

Bec'ause of the relatively smail capacity in the education R&D 

system generally and the continuing need for substantial long-term^ 
attack on educational issues fby R&D professionals, we see no alternative 
to the deliberate use—and reshaping where necessary, as we have argued--^ 
of capacity built up over almost a decade. What should happen f^^^^ ^' 
laboratories is that they must be given a sense of , direction. Th^y. ^^ 
should be staffed, organized, and operated in a way that meets t^he i^i^Or [ 
gram priorities of the major furiding source. We do not mean to impliy 
that the national laboratory ic the only mode of R&D effort, and" we 
turn in a moment to discussing the need in the- next few years of strength- 
ening the capacity of other persons and institutions to do R&D. 

The notion we have of the new or revised institution is ra^iher ^ 
clear to us, and we do not wish to be misunderstood on several k^y 
points: ^ I 

° We do not expect many current university cen4:ers to desire 
the expanded funds and special relationship to NIE that we 
propose, nor do we think that universities Could actively 
involve ortly their best faculty in programs of R&D at the , 
$3 - $4 million level. Thus, while we encourage a review 
of current centers on the chance that some *mlght xseek such 
a relatJ-onship, we do not think a negative decision on that 
score siould rule out other kinds of ties and support for 
university work and training,- such as we have spelled out - 
above. Many other Federal agencies maintain centers of 
research at universities on a scale larger than the indi- ' 
vidual project, without creating the relationship we have 
' spelled out for great national laboratories. 

° The rene^red lab^atories, whether drawn from present centers , 
or laboratories; should indeed span all functions from re- 
search to contact with ultimate users; such a vertical in- ^ 
tegration is one reason for their e^ipanded fundtag. W6 do not 
mean, by using the term "laboratory" whii^h has cfonnoted 
an institution chieiEly expected to concentrate on^ product^ 
dpvjglopment,^ that this function should predominate in ^the 
renewed laboratories. (See our comments later on the whol^ 
concept of development and howit should be expanded.) / ^ 
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Nor do we expect the national laboratories to isolate 
. ^themselves ^rom contact with, and assistance to, the - ^ 

education .system. Good R&D requires a constant inter- 
play of action aad theory, mediated by a process of 
design to translate theory into concrete systems, be- 
haviors, and products. Effective dissemination, adapt- 
ation, and utilization of R&D outcomes will pf course 
require laborat*bries to be in touch with the ultimate' 
users, as well as with linking agents who are helping 
users find relevant R&D. We do not think, however, 
that good R&D flourishes in an environment where for example 
sheer survival of a laboratory requires taking' on a - ' 

dozen small field evaluation projects for local schools 
which are of little interest or stimulation .to. either 
party, or the performance of training in 3 field far ■ 
removec^ from the laboratory's mission. 

We do not imagine that all current laboratories or R&D centers are 
. doing the best job that can be conceived; but we feel strongly that one 
builds on. what exists; ar,d one mov^s, to strengthen that base towards the 
goals of greater quality of work, relvance to the missions of the sponsor^" no " 
agency, and aid to the operating system. ^Casting the present resource loose 
into an increasingly stormy sea of competition for small or short term erants 
seems far from a deliberate .use of the present set of institutions, which 
represpnt such a cost in trial-ai^d-error learning already. We will return 
^co the ..specific issues of how these capacities at the laboratories have 
been -dealt with by NIE. and how our eight-point concept of .laboratories ■'nst . 
described could become the basis for new'NIE policies,' in" ^.he last two chapters 

Local School Systems and State Education Agencies . It is a 
common observation that the operating educational agencies invest ' 
^ittl.e_in research and development, at least as that activity is 
narrowly defined. The 1975_Jataffook offers little information on 
the subject, quoting some tenuous guesses. An earlier paper by NIE 
^^^^^'^ Building Capacity for Renewal and Reform described a tele- 
phone survey of large-city districts which turned up only a small 
fraction of one per cent of any budget devoted to such activity. ' 

. Yet there is more than that happening, an.l we feel NIE must 
somehow build upon the present effort. For instance: 

° _Many states have established planning and evaluation 
, offices using funds from Title V of the Elementary and 
/ Secondary Education Act (ESEA)« 

.State and district accountability schemes jLnvolving the 
anal^ysis of ^tests and other data are multiplying. 

States conduct evaluations of Federal programs such as ^ 
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.Titles I and III of ESEA. ^ 

Some city districts have substantial research and plan- 
ning offices, with sophisticated data systems, curric- 
ulum development projec<ts, and internal dissemination 
efforts. „ • 

^ city and state agencies are iiivolved with a variety of 
outside helping organizations for research, surveys, 
staff training, organizational development, and evaluation. 

We are concerned about two linked aspects ^f tfiis situation. 
Since little is known "about the R&D activity in such agencies, lit^tle 
thought is given' tp strengthening^ their' ability to perform it. As a 
result, there is continued the traditional ^image of the practitioner 
and the operating system generally as the recipient of disseminated 
information, produced elsewhere for their benefit. While this is be- 
ing modified slowly, as developers recognize the need for local adap- 
tation of even the finest tested products, there still needs to be 
a fundamental recognition of the latent capacity for systematic think- 
ing that exists in many systems. 

■ Given the meager resources for educational inquiry and program 
development, it would be an unfortunate waste to write off the present , 
scattered and sometimes unsystematic efforts 6f state e^ndJLocal agencies. 
Of course there are a dozen barriers, such" a3 lack of skill or bias to- 
wards the status quo, that might interfere with' the ability of public 
schbol agencies, to look clpsely at themselves and take remedial action — 
negative responses in some places to the^ Coleman. survey , the National 
Assessment Program^ 4:he voucher Experiment, and other potentially re- 
vealing or provocative R&D efforts make that clear .-^ 

'''As we noted in a "^parallel argument under the discussion of staffing 
for R&D above, it can be said that a present condition of little in- 
vestment b^, school systems in R&D, or even in progr^im evaluation as a 
first step towards systematic inquiry, accurately reflects the incen- 
tives and rewards for such work. While we agree that there never wni 
be incentive fdr a single system to do basic studies ip childrenj^^^arn- 
ing,abwe believe t±at other kinds of localized program development and 
assessment activirtes are useful and proper, and that many systiems are 
workine in this area. The Federal role is to support good examples, 
"Study the conditions of their- success, an^ sul^sidize new versions to 
refine the analysis. As with the argument under personnel, we do not 
think the concept .of school, district,, and state-level inquiry has been 
adequately tested. For a pessimistic argument, but no data,^about the 
pot^Tptial of such a test, see'^John Pincus, Incentives for Intiovation in 
the Public Schools (Santa Monica, Calif ornia: RAND Corporation, 1973. 
publication P-494e. ) • , \ 
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, Nevertheless, we will return in our recommendations to some 
concrete ways in which Federal R&D policy at NIE could begin 'to engage" 
the incipient capacity in local and state education agencies, both ' 
t^.lTnrfT ^''^ directly and to sensitize other grantees aAd con- 
tractors to the need for close work with them. Particularly if we ■ 
are encouraging a more specialized (less service-oriented) set of 
insta^tutional capacities in universities and national laboratories 
we must strengthen the local capacity to do for itself what others' 
may.no longer be doing for it. Finally, on this topic, we want to 
recognize that sorie local school districts have built up admirable 
n^^H^^r^^'' '° f'"*^^ '^^'^ °^ problems and design solutions. NIE 
actt.-M ir^iormed about these and to eru:ourage suoh 

activities wherever they can take root. . 

System Qualitifes , ' / ' 

As a final aspect of the present resourfi^fes for education R&D 
we want to comment on , the degree of "systep^' that seems ta exist. 
We have been impressed, -even though we hav^ far too little"data, to' 
find so little interrelation of parts.. There is an aggregate of in- 
dividuals, institutions, and other resources that combine and relate 
in a variety of ways to a variety of ends. Such an observation- 
strengthens our view that a single Federal agency, with limited 
funds, must not be expected to have massive impact on such a "system" 
m the short run. But it also reinforces our conviction t±at con- 
siderably greater effort must be put into mapping the pieces of the 
universe, understanding the diverse structures and their internal 
strengths, so that deliberate policy toward each part oan be attempted, 
and sa that potential connections and interrelations can be encouraged. 

In this connectioa we note paper's on the R&D system -by Ronald 
Corwin, by Egon Cuba and David Clark, and a thoughtful letter sent 
us by Ward/Mason of "the NIE staff. ^ Corwin describes education re- , 
search as a loosely-knit social system" made up of "disparate and ' 
nebulous communities" of which many are now in an "anomic state." 
Noting that "science tends to be rather disorganized," Corwin re- 
counts the history of research management and concludes that attempts 
to impose more rigid forms of organization have not clearly improved 
the quality of^ the product. «^ 

Cuba and Clark join Corwin in both describing the fragmented 
system and analyzing how poorly paat policy reflects the "reality." 
They suggest an alternative framework to the notion they feel has 

1 . • , ^, . . 

Ronald G. Corwin, "Beyond Bureaucracy in Educational Research 
Management," AERA Division G newsletter, "The Generator," Winter / 
1975; Egon G. Guba and David L. Clark, "The Conf igurational Per- - / 

spective: A View of Educational Knowledge Productiori and Utiliza- 
tion (Washington, D.C.: Council for Educational Development and 
Research, 1974). . 
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reigned .unexamined in recent years, of a mechanistic .system of 
specialized institutions in research, development, dissemination, 
in orderly sequence. They urge that future policy planning take 
in to*" account a number of currently overlooked aspects, of the re^l-^ 
ity -of the R&D system: ^ ^ ^ 

° Knowledge-production and utilization is* a Secondary 
goal for most instituticfns in the system,, and as a 
result the more basic missions of each place will not / 
be dropped or altered in favor of mote systematized 
R&D or other externally-imposed goals 

.° .No institution likes to be placed at the receiving 

end of a system—or at the bottom of a prestige hier- 

f archy~as local schools are in the traditional system 

model - ' - ^ 

* Institutions typically refuse to remain specialized, 
and will expand to take on "related functions, thus 
blurring any permanent distinctions intended by the 
central planners. 

Mason adds that wide diversity of approach is inevitably to be 
found in a field like education which lacks accepted paradigms 
or models of what is important to study and how to do it. 

These points seems excellent advances toward a descriptioA of the 
aggrega^te of people and places doing education R&D,, though we disagree 
with these authors in drawing normative policy conclusions from the 
analyses. ' (For instance, we do not concur that the proper NIE res-^ 
ponse to fragmentation of the R&D Community is to delegate much of xts 
own decision-making to that community.) The iiji^ortant point to be 
•made here is that the system— no in^tter how one comes eventually to 
characteri^.e it, as community, organic body, machine, or whatever - 
is not well-Unown at present. The s.et^ ^ious 

rsKiientrof^poUcy! fr^la^^ encouragement of colleague- 
ship and networking, to central planning and direction, across the 
various domains and types of R&D work, must be based on far better 
understanding than exists. ^ , 

For additional discussion^of the nature of the R&D system, and 
the needs it should be designed to serve, see the paper prepared by 
Sam Sieber of our grou{>, which appears In Appendix A. 

Having said some things about the resources as we see ^hem now 
and how they might develop In the future, we must come to consideration 
of the role to be played by the National Institute, of Education. In 
the next two sections we take up the present context cff forces acting 
on NIE, and thft policy directions being selected. 
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General expectations for an activity like'RiScD, and a set of 
available resources such as we reviewed in the previous section, , 
are twp ingredients for R&D policy. But ^t a specific "time, pblicy 
choices must be made within a confining context of forces, at that s 
moment. Out review of funding policies at the NationaVInstitute of . 
Education w^s called for- at a particular time, because of particular 
( pressures, and we felt the need to inform ourselves about the present 
environment of NIE. Thus in the section which follows we touch on, , 
organized groups. Congress, NlE staff, the National Council on Bdu-, ^ 
cation Research, other p^ts of the executivie branch of the goverranent ,j 
and finally a force from the past—the weight r^f inherited commitments' 
and staff that the new agency- had to carry frbm the outset. 

^ Alternatively, the dis^cussion . could be approached through certain 
issues or questions to which the Institute must have , practical ^nswer| 
and which are the f ocus ^f delsate- and concerh<*.y groups and ' individuals , 
. .within and outside the agency. These include: What should be tbe 
goals of NIE? Who should participate in decisions on what. to do? At 
what points and with how much relative influence? Who should. perform 
J^e Institute's work? What methods are best for doing that wrk, ac- 
cording to what criteria? Who should benefit ftoin the work?; By what 
standard should "benefit" be discerned? How should staff be.arranged 
internally to manage the goals selected? 

These matters have been in dispute almost from the moment the ink 
was dry'-on the authorizing legislation. For some time the basic policy- 
making machinery of the National Council was- not ready, yet th^ staff 
^ needP-H to make decisions. A self-appointed private watch-dog group 
threatened lawsuits over allegedly illegal policy-making procedures 
within months. 3nd the new Director had to try to defend himself agd 
tKe administration in early oversight hearings before the House autHbr- 
izing c0lBmittee. As various parts of the NIE program have emerged ijh 
later months, a variety of others have begun a steady v-plume o£ com-^ 
plaint that they had not been consulted Wd should have been, or that 
the proposed directions were hopeless ia any case. Staff, recruited 
from academia or other. Federal agencies ,<^fmay have hoped to revive in 
the fresh and youthful agency the spirit of educational changfe.-~now 
flagging— that had characterized earlier administrations. TJhey were 
no doubt impatient with lengthy and participatory planninfiTprocesses. 
The fluidity of the early years had the result that hardl4^9n educa- 
tional journal or professional newsletter has not ha* at le^t a few 
, "look-at-the-mess-at-NIE" articles. A vicious cycle of crit^ism. 
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budget cuts, demoralized and lackluster performanpe. and still more 
criticism has only recently abated. ' 

^ « >^ 

We review some of the actors and forces in the drama, as we 
see them today. 

/ 

Organized Groups aqd Associations 

^^^^ 

NIE is relearning the lessons of the years of work leading up 
to the xeglslatlve victory for basic Federal aid to education in 
in 1965— -that a great mafly groups and factions must be considered, far 
beyond an act or agency's inmedlate benef Iclfxles , simply because in 
education politics so many people hW their^ands on Ehe brakes or 
feel that their hand chould he on th\throttlW in preference to others', 

• t . ^ - 

Thus NIE has to deal with teachers and their organizations, 
anxious to be consulted at every step and alert to watch out for 
radical experiments that might disrupt Important parts of their pro- 
fessional world. It must be aware of higher education groups who 
monitor- the balance between, support of unlvjersity scholarship and 
contracts to non-profits and who wonder why little attention is paid 
to their own problems of post-secondary schooling. It myst at the 
. same time contend with organized nop-proflts who, in turn, remind 
«NIE of the poor image that practj^oners have of university research 
and -of their own responsiveness to government demands for new kinds of 
evaluatiearv^nd product development that universities never got into. 
Chief state school officers—at least their leadership and Washington 
of ficials— lost faith in the NIE early on, as a growing role (and 
growing dollars) promised by the DSOE in product dissemination seemed 
nipped in the bud when the work of the National Center for Educational 
Communication was transferred to NIE and the states "were not immediate- 
ly brought in as they had been at DSOE. I^.e chiefs' Council set up a 
special committee to keep an eye on NXE, and played the continuing 
role of skeptic and critic of NIE's budget in the annual deliberations . 
of the Full Funding Committee (setting the lobbying posture of all 
major educational organizations on appropriations) ha^d on their view i 
of NIE's lack of responsiveness to states.. * ■ , 

For their part, scholarly associations' in sociology, psychology i I 
and other disciplines whose hopes were raised by a first-year disclplln^ 
oriented and field- initiated research-grant competition at NIE, grew 1 
concerned when the following year's competition was much more dlrecte^d, \ 
and when no further field-initiated competitions were held. As avail- 4 
able funds ^e^t to existing development activities or directed ^requests- 
for-proposals?%nlversity scholars no' doubt saw these as excessively 
practical activities. The educational researchers themselves have been 
in a longstanding quandary about how much of an activist role to play 
in Washington, thus presenting no united front to the NIE. 

" 

Thus the various organized groups differ in their notions of what 
;^ Q^s proper to be studied, how best to help improve education in the first 
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place, who the proper performers are, and in short— ^about almost 
e^ry key aspect of an Institute's life. A rising appropriation 
mi^t have permit^ted successive approximations to each group's 
expectations, but* that has not been the reality, and to the extent 
the debate is over money, the groups now must fight oyer a static 
budge.t, where someone's increase is someone else's loss. . 

The strongest organized voice, in" the environment at present 
is the consortium* consisting of most regional educational labora-^ 7 
tories and research and development centers (CEDaR, for the Council^ \ 
on Educational Development and Research). TKe lobbying position of 
these institutions is that their unique historical staj:iis as Federally- 
initiated R&D performers, and their present usefulnpss to the world 
of education, argue for a major continuing NIE comn^itment to them. In 
anticipation 6f continuing decreases in their share of NIE funds, the 
group has lobbied vigorously in Congress and With the other -associations 
"for explicit legislative direction to be given tp NIE. to continue their 
work. Pressure on the House of Representatives resulted in a direct 
earmark of NIE's fiscal 1976 appropriation for the laboratories and 
centers, but even greater pressure on the Senat;e resulted in less re- 
strictive language. And, in the service of th^ir undenied self-interest, 
this group of institutions has been almost jSing\Le-handedly telling the 
story of education R&D on Capitol Hill. ) 

It is fair to say that because of the important political ad- 
vantage of the laboratories' and centers ' geographical diversity, the 
present size of their ^budgets (which makes them significant installations 
in any Congressional district), and the sheer persistence of their cam- 
paign on all fronts, the -CEDdR group members have >een the single post 
important' continuing pressure on policy at NIE. They work with Congress, 
with the Council, with staff, and have written and tailed us a number 
of times outside our scheduled meetings. As we understand it, and to 
simplify, laboratories typically claim that their products haVe been 
carefully tested and -enjoy wijde appeal. Research ceijiters argue that 
critical masses of diverse researchers have been broiight together for 
mission-oriented research over periods of time that would never have 
been possible under individual grants and contracts--as was the intent. 

Both argue that the Institute has a Congressional mandate to build 
an R&D system, and that the USOE began that task with the creation of 
the labs and^ centers, with promises of continuing suppott. It follows, 
they would say, >that a clear way for NI| to carry out the mandate is 
by continuing the exiting units as kafitones of a systeW. Labs add 
the political note that continued: f und»t, of their development efforts 
will in tuirn rebound to the Institute's political benefit as consumers 
see more and m(j>.|^ products as fruits pf the R&D dollars. \Thus as educa- 
tion needs products and labs make products (better than others) , that 
; activity should continue at a high level of support. Needless to say, 
these propositions are not supported^ in all quarters^. ^ 
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Congress ' X 

NIE was created in' the House, added to a bill (the Education Amend- 
ments^ of 1972) fundamentally concerned x^^ith creating the new structure 
of Federal aid to students in higher education—a bill most contro- 
versial for its anti-busing riders. Despite the fact that the need 
for a national institute of education had been advocated by a govern- 
mental task force and numerous individuals, with these other issues 
claiming significant Congressional ^ttention^^ the NIE, emerged ^rom 
the legislative process with few backers and friends, and without much 
direction beyond the enormous hopes of the statutory language and 
accompanying reports. This substantive ambiguity, and the huge auth- 
orization X$550 million over three years) probably fueled the hopes, 
which may have to b^damped somewhat, for something for everyone and 
for rapid impifoVements in schooling directly attributable to NIE 
efforts. , Sophisticated statements from a Daniel Patrick Moynihan 
about the need for decades of effort in basic studies before any 
payofr werfe buried in the testimony, not vividly kept before the 
members at passage or since. T 

So without widespread Congressional commitment or understanding 
of a mission, but with potentially enormous expectation^ and a 
shrinking base of Federal 'education dollars to bid for, the NIE has 
run the Congressional appropriatiohs gauntlet, and that story Is by 
now legend. The budget has been In trouble ever since some unhappy 
early encounters between NIE^s first Director and key Membe^ of Con- 
gress. From a first-year "honeymoon" spending level of $142.6 million, 
the President's -budget request for NIE shot up to $188 million in 
1974, but a bare 5^.7 million was granted. The amount dropped even 
further, to $70rmillion in the third year, 1975, against a request 
of $134 million. Even then, the result was a cliff hanger, as the v / 
Senate vojted **zero dollars" and the House reduced its own Appropria- 
tions Committee recommendation from $100 to $80 million after passion- 
ate floor speeches against research by ^ith Green and H.R. Gross- 
As confidence in the agency diminished, report language has begun ±o 
give direction to NIE, culminating in the present deliberations over 
the 1976 budget, where close tt?^ one half the budget was earmarked by 
the House, as the fjrice of granting the modest ^80 million budget re- 
quested. 

Though there have been no oversight reviews by the authorizing 
committees since the early proBing about the National Council appoint- 
ment delays, the appropriations committees\have commented on the sub- 
stance of the program while cutting budget requests. Questions at 
the hearings and the committee reports indicate a general skepticism 
^ about the work under way, its practicality and utility to "those on 
the firing line," and ar strong concern for more dissemination of what 
is already known. Criticism has been voiced, more in private, about 
the incomprehensibility of witnesses, documents, and programs them- 
selves, though in recent months that has been reversed and the Insti- 
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tute has for the first time been praised for its incireased "goal 
orientatioyi" and an understandable, program structure. This histpry 
of criticism and pressure on the budget by Congress had severe impact 
on the agency and has led to reducing the political risks taken at 
every, possible point. 

The relation of the Institute to Congress includes several other 
elements that may be worth adding for a complete , picture of forces 
exerted: 

® ?ome in^iividual Members give strong hints about their / 
interest in seeing certain ^ojects continued, independ- 
ent of their potential for research. ^ 

Some^Members and staf^* on Capitol Hill simply wish that 
NIE would just cleaij:^p its public act, so that clamorous 
lobbyists would go away. Several on Capilol Hill have told 
us, "l^y can't NIE Just arrange some kind of truce among 
these different groups? We're tired of getting, calls and 
mail on this; NIE has "to solve its own problems." 

^ The Institute has been directed to carry out certain 
studies by ACt of Congress. These include a stu^y of 
crime in the schools and a more (extensive review of com- 
pensatory education. 

Some of those most closely connected to the original^ design 
still hope that NIE can live up to its mission for superior 
quality research, done by the most able people available, 
drawn from the wides.t range of fields such as neurophysi- 
ology and nutrition. * 

Finally, we understand that there is in the- background some ' 
continuing interest, in the Senate especially, in another 
reorganization of the whole education segment of the Depart- 
ment of HEW, including the Assistant Secretary, National 
Center for Educational Statistics^ Office of Education, and 
the rest. Public witnesses at recent NIE reauthorization 
hearings made suggestions of this sort also. 

Congressmen typically hear from their chief state school officers, 
local superintendents, and teachers' groups, all more or less tied to 
present educational practice and immersed in today's educational prob- 
lem-— from school discipline to school finance. With limited time or 
incentive to explore the complexities of the role of social resft^ch in 
restructuring the way problems kre framed, it is not unreasonable >f or 
Congressmen to expect to see the NIE trying to solve pressing problems 
of education as'^ they are perceived back home. The inability of th4 NIE 
to coiitaiunicate with key education organizations in the early years, to 
involve organizations in a continuing way in planning, and to articulate 
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a program to educators in t^rms that sounded reasonable, all con- 
^ tributed to the perception' of Congress— held perhaps until recently — 
that "no one back home c^res whether this Institute lives or dies.*' 

In addition to these more typical lUnds of legislative-executive 
relations. Congress is playing another more direct role for NIE. As 
the Institute seeks authoritative statements on which to build prior- 
ities and programs, other acts of Congress are assuming new importance. 
For instance, the Special Projects Section of the 1974 ESEA renewal 
mentioned a number of particular areas of education that needed atten- * 
tion; likewise the Congressional Charter for a 1977 White House Con- 
ference on Education mentions specif i^c items of substance. These kinds 
of Congressional statements are now being taken as indications of 
''national priorities** which should play a part in KLE planning for 
R&D. So, in a general move to be responsive, the NIE is taking on the 
substantive agenda of successive Congressional education eijg.ctments . 

i 

Beyond specific NIE matters altogether, it is clear that the 
Congressional climate of the day or term is an important factor. 
Senator William Prpxmire*s attack on social research at NSF, the ^ 
recent demand that I^SF stop it^ ''involvement with certain curriculum 
projects whic^ some parents and Members find offensive to traditional 
valuesi the further legislation requiring NSE to submit for Congress- 
ional approval in advance all grant actions, and so ton, together form 
a clijnate that is bound to^ have a chilling e^ect on an agency doing 
similar work. 

. < ■ -^t ... 

The N IE Staff * N 

■ ^ ; ^ , . . 

It would be most unwise simply to catalogue External forces 
pressing on Institute policy, as many important d<@cisions and plans 
are in fact in the hands ^^of staff, and their o\m values' and concerns 
are therefore weighty. * r 

^v^As a rule, the staff is younger rMher than older; drawn from 
universities or government rather than |ublic education, critical of 
much current educational practice rather, than satisfied; anxious to 
get pn with their own work and plans ratiier than perfept the plans of 
ethers (either plans made in the past, \ as in the case of inheritances, 
or pla^s raadfe now by outsiders); and divided about the proper mode of 
attack \on education matters (through basic studies of underlying 
processes, through field-based experiments and development, or through 
the spread of current best practice and findings). Many feel they, 
were, promised certain scope'or freedom of initiative when they were 
recruited^ in the first heady days, and some would say the promise has 
never beeti delivered. ^ ^ 

• 

Enthusiasms abound, but managerial talent and planning skill to 
carry them out is varied — as it is anywhere — so that not all staff 
feel well-organized and well--used. It ♦Is not surprising that the 
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Civil Service Commission has found flaws in the loose structure 
of the In5titute and. the frequent shifting of people from task 
to task — which are at variance with accepted government organiza- 
tion and pejrsonnel'^ principles. The result will be a iipre tradition- 
al organization and hierarchy of groups, branches, sections, and 
divisions, which will be at odds probably with such staff values as 
flexibility and which many who are new to government wiU finc^ in- 
hibiting and contrary to initial expectations. 

A former key NIE planner ha6 stressed to us the combined effects 
of youth, desire to make a mark, and inexperience at ^planning and 
managing. As a r^ult, the^first several years saw a, series of 
struggles among camps, each somewhat ^^nreflective and not well-tied 
to outside advice and agenda-setting,^ but firm in /the interest, of 
pursuing a chosen tack-^g^whethef: close studies of /basic processes, or 
more practitioner-oriented work. The Institute's fledgling Office of 
Planning^ and Management understandabl5^"^£ound it hard to bring these 
contentious parties to the table, and to consider in a mature and 'con- 
structive way diverse and soqetfmes^ conflicting efforts, though this 
pattern has Been changing during the term of the Acting Director and 
with prodding from the Council. 

^Thus, in the catalogu^ of forces, it is important to mote that 
staff just as much as outside groups differ on what is best\to study, 
with what methods, by whom, for whose immediate or future benefit. 
Staff differ^ too, in their self-perception. Some wish to be "cata- 
lysts" or synthesizers, bringing together the best minds of the 
research world, drawing aa agenda"* from them and proceeding with 
those parts of the agenda that funds and the state-of-the-art allow. 
Some others, perhaps still in hopes of recreating the typfe of acti>fcity 
they knew at the Office of .Economic Opportunity, place more hope in 
directed work, field experiments^ and close evaluation desij^ned by 
the 'tnstitute to yield desired knowledge and fill information gaps 
for policy-makers. Still others are pleased to apply their skill in 
' answering questions set by Congress in mandated studies such as on 
compensatory ^education or crime in the schools. 

The^at'ional Council - V 

Appointed late, amid Congressional criticism of the Nixon White 
House for the ^elay and the resulting difficult position of the 
Director, the Council has enormous formal powers. With the right 
not pimply •to advise, but to make basic policy for the Institute 
and oversee the program; the Council was s6en by- Congress as a way 
to insulate the agency from an untrustworthy bureaucracy and as a 
balance-wheel in the chaos of fads in education.^ 

Since its swearing-in during July of 1973^ the Council has met 
sixteen times (though not always with a quorum^, and has come together 
in ad-hoc sessions for visits or discussions/with other groups many 
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Other times. This is a rapid education process, and one could wish 
that the terms of office were somewhat longer so that the experience 
gained by Council members would be available longer. And yet, despite 
this admirable record of meeting and conferring, the Council remains 
a rather invisible force, certainly not known among NIE staff as a 
source of policy that affeots their lives; ^nor in the wider R&D or 
education worlds ip it readily acknowledged, we sense, as a shaper 
of the young Institute. • , 

The Council, vre understand, feels that that it cannpt, and indeed 
shoul^not, act without close relationships with the Director , that 
their proper role is to press for staff analysis of options, honest 
reports on outside views, and for recommendations from the Director. 
The suggestion,, by CEDaR, that the Council needed its own staff to 
counter potential biases or specisd pleadiitg Inherent in NIE staff 
work, was rejected summarily'. In fact, the Council has gone on 
record expressing satisfaction with the present arrangement, in ' , ' 
which one NIE staff person gives su^tantive help to the Council and 
several others provide loglptic'^support. -(Even within, the .Council's 
preference not for independent staff work, but for .assistance in 
coordinating the work of NIE staff in policy-development., there seems 
to us to be fullt^me work fqr several people.) 

How has the Council been an influence, then? There have been 
three ways. First, continuing stress by the Council on its needs 
for options to review, and continuing pressure for coherent presenta- « 
tions by top staff of how directions were being set,' have had strong 
impact on internal NIE processes. Staff who have no idea what the 
Council does will nevertheless recall a much more formal and elaborate 
planning sequence leading up to the 1976 budget, than had ever been 
seen before. This systematizafion was in large part a direct response 
to pressure from the Council. . ' I j 

Second, and contributing to tl>e Invisibility even more, is that 
the Council has influence through a few top sta'if members who 
know them, most intimately , as these' NIE leaders internalize the views 
of the Council. Thus, without hearing the Council explicitly resolve 
upon something, a Director or other top official may himself reject a 
course of action because "I know the Council won't like that," or pro- 
ceed in a certain way "because it's consistent with Council thinking." 

Third, pf course, the Council does make substantive decisions,, 
such as setting priorities and reviewing budgets. They oppose^expan- 
sion of the voucher demonstration activity, for instance. They, directed 
that 1^^15% of the budgetj^e allocated to field-initiated bafeic re- 
search. The 1976 budget received extensive scrutiny, and vairious pres- 
sures for and against programs were explicitly weighed and resolve^. 
Yet even here, in the last two years, we are told, there have been times 
when, private Cbuncil decisions have been announced publicly as staff 
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recommendations, then endorsed by Council resolution, so the source y 
of policy Is not fully clear. . 

^ Examples of Council action on R&D funding policies include: 

® December 1973: Establishing five priority areas for 
future emphasis, an^ji allowing the Director to defer to 
another year some obligations to free up funds for new 
work in the priority areas. 

January 1974: Making qlear they expected NIE staff to 
solicit public opinion is a normal part of the planning 
* process and to reflect ihis opinion in material presented 
to the Council — in contrast to the isolation and non-^ 
consultation they perceived. ' ^ 

® September 19745^ Obliquely endorsing the direction of 
» ^ diversifying the set of R&D performers, by supportiiffe 

the Dir^ector in trying to bring current work to an end 
" withhideful results, in the current year (Fiscal 1975) 
and Wjging that planning for FY 76 give particular atten- 
tion t:^ helping institutions align themselves with NIE 
" program purchase policies. ^ 

^ Octobef 1974: In anticipation of Congressional dction 
cutting the FY 7 5 budget, the Council endorsed' the idea 
of reducing lab and tenter work by 15% overall, based 
on project-by-project review. 

^ January 1975: Stressing that policy ^oals used by the 
Council in reviewing the 1976 budget also be used in 1977 
plans — including targeting funds to state and local agencies, 
stress on competitive processes, public involvement in 
planning, and programs responsive to specific needs of 
education. . 

° May 1975: Directing that the Institute strengthen its 
emphasis on evaluation of R&D products and the capacity 
of R&D performing institutions, and asking that the Director 
return with some plans for action along these lines. 

The last example is of particular interest to us, in view of our 
own strong views concerning ways in which NIE should be more aware 
of institutl^onal capacity than it has been. ^ 

In some other ways in recent months the Council may be organizing 
for more effective action. We note that It has formalized its staff 
office, and in addition ha|.^^^ up committees of the Council 
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fpr specific functions. An Executive Committee wllX give continu- 
ing oversight to NIE Implementation of Council policy, and will 
provide a manageabli forum for considering new policy Initiatives-, 
to be recommended to the full Council. A committee on Program 
Development and on Strengthening Education R&D cah look at the , | 
substance of the various NIE activities and consider whether the 
mix of activity is responsive to continuing education problems. 
This group, as well as the third, on* Council Reports , could be the^^ 
forums for consideration of the analytlq work to be. done by the R&D 
System Analysis unit we later recommend NIE should establish. 

Since we argue that the reqbired reports should be seen as im- 
portant opportunities to tell the stgry of education R&D, we" are 
glad to see the Council formally establishing a group to atten<J to 
these tasks., 

Regulators Elsewhere in the Executive Branch 

The forces we have been talkin'g about have impact on both pro- 
cedure and substance, such as the Council in effect directing both 
the process and nature of. plans by the questions it asks. Xhere is 
in addition a set of forces- acting on purely the procedural conduct 
of NIE business which concerns many staff, and may be worth commeht- 
ing upon briefly. / ^5ibs„ 

Outsiders to government often suspect the "interf erenceV -of the 
Of flee •of Management and Budget (0MB), the planning and analysis of- 
fice chiefly cC)ncerned with the President's annual budget. The agency 
has grown in recent years to have oversight of many more aspects of 
government ac^'tivity than the budget, in the name of coordination. 
Typl'cally, however, OMB has not Intervened in the Institute budget 
process as It was famed for doing in earlier years of the 1970s in 
cutting, other social programs; NIE is seen as an administration 
initiative, though not to justify an enormous battle with the 
Cpngress. The "forms clearance" procedure, carrying out the require- 
ments of the Federal Reports Act that all data-gathering Instruments 
be screened, by OMB, is often decried by researchers as potential 
government censorship, but in fact sedms not to be so exploited by 
OMB. * 

Rules and regulations concerning employment, 'advancement,, and 
organization oiE staff are made by the Pepartment of HEW and the .Civil 
Service Commission, and these, too, are seen by many at NIE as lnhlb^.t 
ing and irrelevant. As mentioned above, staff hdve recently been .re- 
organized, and much more exacting and rigid job descriptions hsAre been 
ordered for each employee, in the wake of, a critical review by the 
Commission of past personnel policies. Staff perc^lVe that the Com- | 
mission views all agencies alike, and would, staff and organize a check 
writing bureau in the Veterans Administration the saAe >^ay as a re- 
search team\at NIE~wlth obvious dysfunctional^ consequences . Whatever 
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the merits of the argument, the fact remains that NIE's hiring 
authority for certain positiops has been suspended, the Institute 
has another black eye, and flexibility of action is reduced for . ■ 

the future. ^ \ 

The general political climate of suspicion of education- 
related and research-related grant and contract procedures, 
suspicion encouraged for years by/ now-retired Representative 
Edith Green, leads many business administrators in the Department 
of HEW now to interpret procurement regulations in the strictest 
ways, we are told, with the result for instance, that NIE seems to 
be virtually precluded f roi^^considering unsolicited proposals that 
are not part of some formal competition.' The episode last year of 
3 disgruntled contracts officer from NIE publicly spreading a 
mlicture of fact and fancy about' "scandals" in NIE procurement on 
radio, to Jack Anderson, and anywhere else he could find an *ear, 
with the inevitable 'resulting General Accounting Office inquiry 
lasting for months, has had a jchilling effect. 

It isVincreasingly difficult to involve groups of outsiders 
formally in the planning a^^ review of the Institute's program, 
some feel, just as that.'Ifecomes a more central tenet of>policy. This 
problem result's from a Federal Advisory Committee Act and additional 
Department policy which lay strict rules on ^Jfeicially-constituted 
advisory panels, and conflict-of-interest rulings by NIE and Depart- 
ment attorneys which inake i^ very hard for anyone associated with 
research to declare him- or herself free of possible benefit from 
the advice to be given. r f 

From the parent Department of HEW itself, ineiuiJing the Secretary, ^ 
the various staff offices serving him (Planning and ^E^aluation, Budget, 
Administration and Management),^ and the Assistant Secretary for Educa- 
tion who presides over both NIE and USOE, there seems not to have, been 
^any remarkable degree of general programmatic regulation or oversight. 
Again, as an Administration initiative, the NIE has been an object of 
concern, dismay, solicitude, but no constant or overt meddling (nor 
grand gestures of pride, pleasure, or support). The Secretary and the 
Council jousted over expansion of the voucher program, with each finally 
tacitly agreeing they coujd veto the other,^S^ a compromise was worked 
out between the Secretary's eagerness and the Council's reluctance, to 
ftind some further planning. But aside from that particular episode, 
however upset the Department may have felt at either the substance of 
the NIE's work, or its political ineptness, little action has ensued. 
We do conclude that ^he new Institute has needed flexibility to operate 
in areas such' as procurement, which has not always been understood or 
sympathized with at the Department level. Ahd we urge that for both 
that purpose and for others, the ties between the -Secretary 's office 
and the NIE be examined and strengthened. •Strong support will especial- 
ly be needed from that quarter if the lab-center review process we have 
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• in mind is to be successfully implemented. 

The Effect of Past Actions as Determinant^ of Present Possibilities 

A final feature of the %^text that we must note is the crushing 
history of inherited commitments that we hear about repeatedly from 
NIE staff. The Institute, the;y wish us to recallj di*d not start fresh, 
and was not able to build its program -as it chdse^. It was "given" a 
great, many staff and programs from U^E, OEO, and perhaps even else- 
where, aespitTe the Congressional intention that NIE avoid many past 
mistakes of research management. Yet the early budgets were entirely 
filled up with such items, contractual commitments thjat had to be. 
^honored, of enormously varied quality^ud purpose, though to be sur§ ' 
the Institute had signed many of the contracts itself*. 

Thu^, in the first few year.s, goals could not be set a priori, 
performers could not be chosen against goals, nor new constituencies 
be imagined; all came with the territory — projects, project monitors. 
Implicit and explicit goalte, performers of R&D and constituencies who 
believed in the activities,^ 



If appropriations had risen as everyone expected, the inherited 
ork would not have bulked so large. But the In^itute's powers of 
persuasioli did not match its appetite, and under a steady budget the, 
weight of the past has been felt more and more. And, of course, one 
cannot build new constituencies when almost- alJL funds are tenaciously 
being retained by old constituencies. The mobd must have been schizb-. 
phrenic in the first years, as NIE was expected to quickly demonstrate 
new directions and plans, but was also 'asked first to "show what you have 
accomplished with your present funds." . * 

We caii understand, too, the sense of fairness which may have been 
felt to prohibit differentiated dealings with laboratories and centers 
in ^he early years. Three-year contracts had been signed in many cases 
for specific programs. To' change course in mid-stream to a more''insti- 
tutional relationship with some^, everf as several ^^lE programs might have^, 
wanted, was seen as impossible' because of the general commitments made 
to a program-purchase policy. Even so, this history hardly explains 
why'there should be such dearth\)f policy thinging about the institutions 
now, at the end of the three-year commitments. 

Conclusion ^ 



4 



From groups. Congress, staff, and past commitmentis, then, the NIE 
has available not so much a mandate but a contradictory (and often 
mutually exclusive) set of directives as to basic goals, means to attack 
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them both of method and performer, rights to participate in the 
process, and expectations for the eventual result against whi^h 
to measure achievement. And froiti the standpoint of basic re-^ 
sources, '^In^titute managers have scrambled to preserve some in- 
eternal freedom to initiat^^ the face of heavy aggjtegate inherit- 



ances and heavy outside prSisure to constrain the budget through' 
earmarks. * 

• ■ '^^ o . ^ I : . • ■ 

In this contf.|eiX't^ tt>e ^o^e basic and fundamer^al dilemma of 
research managers thait ve.J<i^not envy is resolv^^the competing 
pressures for scientific activity vs. constltuer^y^uilding. Since 
the political muscle of the oicganized basic^sciende community — 
those for whose support and nurture the Ins^tute was in part cre- 
ated — is tiny, the. alternatives are closer ties^o the concerns of 
the service-delivery system and its powerful constituent associations, 
and/or the organized R&D^ institutions in groups such as CEl)aR. Both, 
such moves have definit>6 consequences for the range of acceptable 

in support. If the burden of makings up ,a 
eacjih year) seems too impossible under this 
pressures, ,;Lt may become more and more ' 
Tiding research through "set-asides" or 
irmarked fraofei^s of the funds appropriated 
for educational service programs. For in^ance, research on comp- 
ensatory*" education -e^uJIjd be funded handsomely with a fraction of a 
ger cent of the appreciation for Title I of\ESEA. Of course, ^ 
pre&sures would still be brought to bear on^^^ctual spending decisioris 
made by the agency, but at least the basic annual funds might be less 
vulnerable and problematic. The constraint in this case would be 
that the Institute's activities would be much more closely tied to 
the substance of legislative enactments of the Congress. 



activities the Institu<:fe 
politically viable pr/grrfj 
set of mutually exclusivl 
attractive to think of fi 
permanent legislatively-€ 



Within this set of forces, then, NIE is taking certain policy 
directions in its Fiscal 1976 budget,^ and in ^kfe 1977 plans now in 
process. These direction^ include focus on various tasks or problems, 
to be done in various ways by various personnel, and procured in 
various fashicAis. We turn next to the R&D funding policies that are 
our major focus of assessment, analysis, and recomnjendation. ^ 



V 
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POLICY DIRECTIONS AT NIE 

1 



This consultant group was convened out of concern at NIE to 
have an outside look at its policies towards a set of institutions — 
the regional laboratories and the R&li centers. In our view, the 
direction for such institutions should result from two sets of anal- 
yses and policies: ' on" the one hand, general NIE program planning 
and design — choosing substance t(f work on and making further choices 
as to strat^egy; and on the other hand, close understanding of the 
various resources available to carry out the kinds of work implied 
by the planning process. With these two kinds of information well 
in hand, policies pf procurement can be tailored to fit the state of 
the community of performers who might be capable of doing -the job, 
and subsequent policies of ><management can similarly be fashioned 
depending on the work and the worker. General issues at the level ^ 
of "for or against institutional support," or "loose or ti^ht moni- 
toring," must be -raised in more concrete contexts. ' ' 

The following discussion focuses first on the current Institute 
policies of program design, planning, and substance. Then we review 
the ways that resources seem to be found and linked up with the In- 
stitute's program. Third, we discuss specifics of , the funding pol- 
icies toward laboratories and R&D centers. We conclude this section 
with a discussion of a new role NIE should take on, of understanding, 
analyzing, and giving leadership to the whole R&D system in education. 
. V . ■ ■ , \ 

We should say at the outset of this review of NIE pojticies- that, 
as noted in the^preface, we emerge from our work' stronge^ supporters 
of the NIE concept than^we were at the start. We believe in the idea 
of such a lead agency, and we appreciate the efforts of staff, grant- 
ees, contractors, and outside groups to keep the agency going in the 
first three years. However we do^ not accept or support NIE completely 
as it is, and we see a number of essential changes in policy, practice, 
and general outlook that must take place for NIE to fulfill its pro- 
mise. ; * 

We, prctbably along with ma^y other distant observers, had been 
aware before taking ^this assignment mostly of the bad news, the poor 
publicity the NIE had been/receiving ^in almost every quarter. Yefe* 
with the opportunity both-^t^Klook more closely at ^me of the com- 
plaints, and' to learn a great ^i^l more about the Institute's staff 
and directions, we find much in the recent months to fejpl positive 
about. 

w 

° Begun by the Acting Director and continued in the 
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recent weeks by X^^e new Director, a serious and 
/ sincere effort to strengthen relations with various 

constituencies , Congress, the teaching profession, ' 0 

state departments, of education, and othersi,. 

° Greatly Improved responsiveness to Congressional in- 
tent, in the presentation of the Institute's program 
and in carrying out mandated studies. . ^. ' . 

Serious strategic thinking done in preparing the Fiscal 
1976 budget, »,.wlth hard choices made and priorities set. 

Revisl^on of the agency structure to emphasize substan- ^ • 

tlve areas of work, rather than functions such as re- 
search or program development which h^ve little meanlng^^ ' 
^o the general public. 

° 'Establishment of new committees by the CouncTl for more n 
extensive work, and an Executive Committee for further 
work between meetings of the whole.. 

° Opening of NIE planning and policy-making more and more^ 
to the scrutiny of outsiderg^j^^Aich as with' our own group, 
or the group which spent two da^ reviewing tentative 
program plans for Fiscal 1977. # 

° An Increase in Internal communications through newsletters, 
more frequent colloqula, semlnajisfpresentations by guests, 
J, and other means, to reduce.^.=fe!fe isolation of programs from 

eaffch other affd rebuild a/splrit of productivity among staff. 

° Clarifying to some extent Internal planning and decision- 
making processes. 

Some complaints from the l^ield seem sometimes to take little note of 
these things. We have occaslon^ly felt we were hearing anecdotes from 
the first year or so of the agency, -told and re-told without charity, 
historical perspective, or recognition of the present shifts* 

Planning and program design ^ ^ 

There are three policies of this sort that we* turn to first, as they 
result in the substance toward which funds are directed. They are: 
defining the agency's purpose in terms of wor£ on six specific education 
problem areas ; Involving a wider range of groups, individuals, and or- 
gan\zJtions in setting the agency's agenda; and better balancing of the 
actual>^actlvities supported, Between payoffs in the long, medium,- and ^ 
short run. We take them up in order. 

Problem focus . I^i part reflecting "priorities" identified in 1973 but 



not empljasized in budgeting before FY76, the six problem areas chosen 
as the budget and organization structure in hate 1974 are as follows: 

* Improving the connections between education and -wdirk; - , 

® Finding ways to remove barrier's to education caused by 
racial, language, or sex prejudices or stereol^es; im- 
proving the equity , of treatment of participants in educa- 
tion generally; • 

® Improving the gapacity of schools, school districts, 8hd ' 
community groups to solve their own problems through in- 
quiry a*tid innovation; - 

^. • Improving the productivity of education through strengthen- 
ing, financial, management, and technological tools; 

® Managing' an increasing base of data on research, products, 

and good practice, in order to disseminate it, and helping 

states and school districts to gain access, to the base and 
link* it to users; * - 

Strengthening the ability o#;the schools to teach basic 
skills. ^ . ' i . ^ ' 

We support strongly the oproblem-focus and regard the problems 
chosen as significant and needing attention. We find such s'upport 
rather easj^, however, since we share the perception found many times 
over in the field, that the ^eas are "broad enough to drive a Mack 
tr^ck through" and can encompass most of the major education issues 
one could conceive of. (If anything, we would welcome even sharper 
definitions of NIE tai^ets, based on assessment of areas where the 
strongest impacts coulfi be made with limited funds.) But it may be 
very useful to have such foci built in for a while, to be able to 
answer the endless petitioners for funds for every imaginable scheme 
that "our appropriation does not cover activity of that (other) sort," 
and to be able to tell Congress just what the NIE l^s up to. And as 
presently-supportedj^institutions wish to know about their future role 
in the NIE program, it can now be clear that major funds will be spent 
only on activities within those proh|[em-areas and that no othen 
fields of work can be supported to ^y substantial degree. In the 
climate of tight money and demands ^r clarity now pressed on NIE, 
we see no alternative to such a policy ^of focus, no way that NIE 
could spend major funds, for example, simply according to scientific 
disciplines" (a program in sociology or a program in psychology, for 
example) . 

We are concerned about the substance of the chosen areas as well. 
For it seems to lis' that the probliem-f ocus of the 1|976 budget may not 
in fact reach k^y problems. For instance, it is not clear to us what 
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the overall NIE effort will be in the- areas af education of the 
disadvantaged and problems of inner-city schools, eithei; in pro- 
gram content or resources. At least none of the budget materials 
and analyses that we saw allowed us to piece together from the 
4' presept problem-focus how activities' would relate to these special 
needs. As another example, we might ask about concentration on 
certain elements of the educational program in the country, such' 
as the junior and senior high years, which have been the focus of 
considerable debate at a highly abstract level in recent years bu't 
which CQUld be followed up In cot\crete plans and designs with profit. 

In view of our concern about gaps in the coverage of present 
problem-areas, it seems to us that in developing plans for work 
within each problem-area, some more formal advisory or intelligence ' 
system could be at work to insure that the definition of the problem 
stays "real" and current with the state of, the ecfucation system. 
Perhaps some common "health of educati^" indicator projects with 
the National Center for Education Statistics could be mounted to 
keep an eye on broad trends in.each, problem area. 

We agree with the Council that NIE should not be held account- 
able for solving each of the problem^ it identifies for attack — 
education and work in the present economic situation is a clear 
case* But we sense that "there is no long-range planning process 
either for changing the mix of activity within a problem-focus, or 
revising the overall set of problems. Most other agencies haVe 
equally, if not more, primitive processes for planning social R&D, 
so we do not mean to imply some high standard that NIE has uniquely 
been unable tQ meet. And indeed, the creative adaptations by sev- 
eral NIE programs of the National Cancer Institute planning process 
are to be applauded as very useful experiments which deserve study 
in their own right. However, for the future, and especially iabove 
the program level,, we sense that there should be an equally creative 
planning function, beyond that, for crisis analyses and annual budget 
reviews. Perhaps there should be a separate staff^f two or three 
people within the planning office who are well-re^ and catholic in 
their interests and contacts, with a few resources to convene people „ 
outside of current program areas. Hopefully such a staff could listen 
closely fo users of R&D, to bring back to rhe NIE not what can be done, 
but what should be done, when time comes to reconsidfer the prob- 

lem areas for NIE focus. ' ' , 

Broader involvement in setting the agenda . The Institute has a 
policy of— now seeking broad participation -in the design of its work, 
and such a claim is regularly made about' the development of the chosen 
problem areas. But the choice of such broad headings is really only 
the beginning,' and the subsequent steps of choosing sub-priorities 
and strategies need to be as open in process, and as credible in sub- 
stantive result, as the first step. We find concern in the field on 
both counts. 
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As to process, we heard repeatedly of anger at unmet expectations 
concerning the openness of planning. Scholars and practitioners, in 
associations and institutions, told us of promises^ made and broken, 
**involvement" that was neither serious nor sustained. A teacher group 
especially forcefully reminded us* that? occasional meetings in Washing- 
ton of high-level staff are no substitute for serious participation 
of practicing -teachers in the work of ^lE grantees and contractors 
from the outset. With all the talk about a new emphasis on outside 
views at NIE, it was disturbing to hear that any group needed, as 
these teachers did, to "break doivn the door" to get to the table at 
NIE planning conferences of significance to them.. 

We heard that the problem areas were chosen in part because they 
covered a significant fraction of the present work NIE has under way.,^ 
But if this is true, At is further disturbing--again on the subject of 
the process of planning — to hear as we did repeatedly at laboratories 
and centers, as well as at other R&D institutions, that top-flight 
staff at such places with ex1:ensive background in relevant work, feel 
di^stant and uninvolved in the planning and designing of future work in 
their areas of competence. 

We recognize that people who complain about being uninvolved may* 
in fact be seeking only approval for continuing what they are present- 
ly doing, rather than analyzing a situati^ and entering into a pl^n-^ 
ning process. We recognize the several agenda-setting conferences 
that NIE has held, with massive participation of many interested groups 
and individuals. Thus, we cannot judge the factual accuracy of these 
claims about participation. But even the perception in the field — 
not confined to the beleaguered labs and centers, though as appropri- 
ate for them as anyone else--that NIE is aloof and hostile,, will be^ia 
"^continuing problem in constituency-building and drawing on the full 
capacity of the R&D system for NIE work. \ 

Our own contacts with the NIE staff have shown them to be b/ight, 
capable, hardworking, and far from the stereotype of the Federal 
bure.^ucratic timeserver. However, we feel that their very articulate- 
ness,Wand their concern to be involved with the work they oversee, 
may ^pear overpowering and arrogant. Whatever the shortcomings of 
resealrch management at USOE, the research field may now look back on 
those days of laissez-faire as a golden era of non-interference by 
the government as many new employees at NIE carry out a more direct- 
ive style of research management, and not always with t^e research or 
practitioner credentials to legitimize such a role. 

Reducing long-term development commitments . "As a third element 
of planning and program design, we note a policy at NIE aimed at \ 
balancing long- and short-range activities more equally in future 
budgets, NIE argues that the 'mix of activities has in the past been 
too heavily weighted by large-scale, long-term contracts for "develop- 
ment" work, chiefly at laboratories and/ R&D centers. The NIE argues 
in its 1976 budget ma^^rials that pro§5:am plans reflect a move away 
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from such a concentration, to allow for shorter-term payoff activity . 
in dissemination and in policy-studies for decision-makers who need 
information soon, and to allow for reinstatement of new basic research' 
awards which were cut off in 1975 when Congress reduced the overall 
budget. 

These new directions may have several positive features — politi- » 
cal appeal, substantive merit, a^d responsiveness to Council policy. 
But we have reservations about this policy, to the extent we under- 
stand it. Either we do not grasp the reasoning against ^^'development" 
in general,. or we do not see distinctions among types. So far as 
developed products from R&D centers and laboratories chiefly in- 
tended for teachers and childrep, we did not gather new data\gpf 
quality and impact, nor was NIE's oi^m survey data on products yet 
available, for us to judge the quality of this type of work. Several 
of us who pressed staff of labs during visits or in correspondence 
, concerning effectiveness remain unimpressed, and we understand tha't 
the National Couniil shares the general viex^ that contractors have 
notgthad funds or /other incentives to gather extensive, long-term data 
on product impact. Yet we^ liked what we sax^; at a number of places, 
where unique and useful materials seemed to be under development and 
dissemination. A nu^aber of our correspondents agreed, and our tal^s 
with leaders of the National Education Association, American Associ- 
ation of School Administrators and the American Educational Research 
Association showed further consensus that some ^aces x^ere doing do- 
ing development work of great promise or actualv^complishment. 

We recognize that somfe NIE problem areas will continue to support 
lengthy development activities , of various kinds. And we recognize 
further that it is quite proper to question the continuing need for 
larg(^-spale curriculum projects, for the vogue of such efforts has 
somewhat passed and teachers are often asking for different kinds of 
help than just through materials and media. Also, publishers have 
increased their output of diverse and attractive materials, some 
even 'as the result of field tests and including built-in, objectives 
and assessment devices, all of which used to be the more exclusive 
hallmarks of Federally-supported laboratory products. 

»f 

However, ther^are other kinds of "development'' that seem to us 
to have good claimiron NIET resources, and which should not be over- 
looked in a rush to get out of an expensive type* of activity. We have 
in mind, for example, development of models of in-service teacher train 
ing: system-change development work, such as rethinking compensatory 
education altogether as NIE has been asked to do by the Congress, but 
which could have been initiated by NIE if it h^d not been* asked; dev- 
eloping post-elementary school models that incorporate some of the 
thinking of the past few years aboiit the fallacies of the comprehens- 
ive high schooll^ developing alternative concepts such as the employer/ 
experience based career education projects; and developing new decision 
making and communications systems between school and toramunity, or 
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between students and administrators. These are just a few examples 
of education program development, beyond curriculum materials , that 
seem worth exploring, and we hope that there has been no decision 
against such projects. We of course understand the need for havinj^ 
work under way that will result in tangible results in, the. near 
future as opposed) to projects- requiting f»ur or five .ore^^ars. 

As a final note on program planning and design, we understand 
that NIE expects to reinstate a policy of supporting basic research 
in various problem areas to a total of approximately the Council- 
pandated 10-15% of thft^udget. We hope that NiE in so doing will- 
not return completely to the earlier pattern of scattered, uncumu-* 
lative single-project work by university and college scholars. 
Vfliile some funds should be available for promising scholars who 
have not managed large projects (or do not wish to), we also feel' 
that investment in longer-term efforts and sizable groups should 
be encouraged. Under that pattern, social a^tems can form, the 
group can grow and develop, adapting its approach as the problem 
becomes better known, and training some students along the way by 
apprenticeship. We do not underestimate the difficulty of getting 
useful research out of groups of university faculty, and as we noted 
above, the R&D center must be only one among many strategies for 
attempting this feat. Nor do we place sole reliance on limited 
studies by a professor and an occasional helper or two. We hope 
that as NIE seeks "better balance" in the program mix, granl;;^ of 
any size will be made after serious thought as to the state of 
particular fields and the potential for significant advance through 
the precise type of award being considered. 

Finding and Supporting Those Who Can Best Do NIE's Work 

We understand there will be in effect ir>.;1976 a policy to diver- 
sify the performers of NIE's R&D activity, with open competition -as 
the strategy for this widening of the net. We agree that there are 
untapped resources'^ in university departments including education or 
in the publishing industry and underused resou'cces in local and state 
education agencies. -But we also haye concluded that there is not any 
excess or over-abundance, or even adequacy, of R&D resources, especially 
not resources of high quality. So we feel it is simply not true that 
one needs only to advertise the NIE's needfe in order to have immediately 
an array of qualified people or organizations to choose from. We dis- 
cuss two aspects of linking ap with available resouixes: fijst, knowing 
present work, and second, drawbacks to the pjresent style of compd^tion 
practiced at the Institute. 

/ ' • . ^ . 

Hoping- to find some new friends, new constituencies -for R&D, and 

hoping also to start fresh projects that could avoid the discredit of 

the old, it seems likely that NIE staff in its first two y^^rs had 



incentives to vievr inherited projects and their performers as 'unim- 
portant-, of .inescapably lower quality than anything new. Indeed, the 
presumed 'shortcomings of the past research establishment had been a 
chief reason for wanting a new agency. Thus the familiar '*'not invented 
here" syndrome may have "justified" not^'giving inherited work close 
attention or investment of time in review and management. Recall that 
budgets were to rise, which meant large plans on the drawing boards, 
consuming most available staff energy. Some of us heard repeatedly 
from'project officers that "no one hacl, taken a very close look at 
the work on this project in the first year or two after we got it." 
Such comments were mirrored in our afield visits and questionnaire 
responses, as managers of work at laboratories and centers recalled 
their projects being shuffled from office to office, with no project 
monitor in place long enough to become familiar or effective. 

. ; The situation seems to have improved, and these same project of- 
ficers in most cases had been with their charge's for some time, though 
travel allowances to be with projects in person seemed inadequate' for \ 
some of the larger efforts. But it troubled us most that individuals 
clearly were operating in a vacuum about how to work with projects, 
how closely to be involved, how much they needed to know to carry out 
what sort of role. Such a situation, of extreme di^vers^ity in how much 
is actually known about specific work in the field, es^iecially at lab- 
oratories and centers, makes one skeptical about policy changes that 
should rest oxT judgments of present activity. ^ 

' Our second concern about the way NIE juses its resources is its 
apparent policy of procurement^by contracts awarded after open compe- 
tition in all but excepc!k)nal cases. ^. (We are aware that in the last 
t^o years, owing to tight^budgets, many procurement actions were sim- 
ply continuations of grants and contracts awarded at the outset. Nev- 
ertheless, th.e intended policy direction for th« future, and for those 
new f unci s let out in the recent past seems^to us — and to matiy in the 
field — to be towards open nat;ional coiopfetttions, notwithstanding sever- 
al exceptions such as the Problem- Solving competition that we were told 
about repeatedly.) This emphasis on competition was recently reinforced 
In the Senate appropriations report for FY 1976, and was put forward 



NIE's own data show a preponderance of funding through contracts. 
In FY74, 497 contract actions accounted for $47.6 million; 129 grant 
actions accounted for $14.6 mill#ti. Through April 1975, 204 contract 
actions resulted in award of $44.4 million, while 18 grants had re- 
sulted in $5.4 million of awards. The choice of^ instrument has many 
interesting and important ramifications that we do not go into here, 
but which need exploration in the NIE coi:>text - including implicit 
incentives to use contracts because of the 10-15-month delay in i$su- 
ing regulations for grant awards, even though such awards can be made 
with much more flexibility of procedure than contracts. 
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Strongly in the Background Paper and in other NIE documents. It 
may. be politically important just now to hiave a policy like this 
to counter lingering criticisms of U.S. Office of Education i/rocure- 
ment of social research. But we feel it has serious drawbacks, 
both intrinsically and as presently used, as a tool for any, particu-' 
lar deliberate action .such as diversifying the performer #ix. 

^ As typically executed, the open competition based on an agency's 
Request for Proposals (RFP) has at least the following problems: 

° It is not always true that agency staff can write clear 
and useful specifications for what is wanted, particularly 
if, the work stems from a planning process where the agenda 
was drawn up by national experts from outside the Institute. 

° The costs of bidding are -eventually added to the govern- 
ment's cost in future procurements, so a high rate of 
bidding and' the accompanying high rate of unsuccessful 
proposals is in the. long run drawing funds away from 
'performance of the work. Where there are only a few 
good performers for a given type of work, the rest of 
th^ competitors have little chance, and their costs of j 
failure are a drain on energy and time that might have 
been a-^^^ided. Further, the cost to the government of 
reviewing a great many proposals is not always reflected 
in superiority o£ the final product as compared with the 
quality obtainable under more limited competition^. 

• i . - 

° Extensive competition amonjg a small number of organiza- 
tions capable of large-scale work in education R&D may 
tend to promote disintegration and professional secrecy 
within the group — negative results to be avoided if 
possible. 

° Where proposals are judged by Institute staff, the current * 
procedi!ire prevents them frgm working with proposers to look 
at ideas or to review advance copies of proposals so as to 
avoid submission of obviously unresponsive or unqualified 
ones, or better, to strengthen marginal ones. 

Researchers who e;}camined the competitive process in a slightly dif 
ferent field, social program evaluation research, commented how compe- 
tition failed to produce quality, and in fact quite the contrary. 



A. Biderman and L. Sharp, The Competitive Evaluation Research In- ' 
dustry (Washington, D.C.: Bureau of Social Science Research, 1972), 
p. 46'. 



ERLC 



-56- 



In addition to the above-mentioned^barriers to quality^ and 
hidden costs, the strategy of open competitions as a device for 
bringing to the attention of the NIE all those who should b^ con- 
sidered for research funds rests on questionable" assumptions about 
th^e market. First it is assumed, explicitly in the Background Paper 
on o "Labs and Centers ," that there are- now abundant resources for edu- 
cation R&D as -a result of the heavy demand for social science evalua- 
tion and advice in the 1960s. We feel, on the contrary, that this is 
a misplaced hope, and that a few hundred consulting firms do not an 
R&D system make. Second, reliance on competitions assumes that those 
who would be good performers know about the solicitation and are able 
to enter the market. Some challenges to this assumptiorl, at least 
based on present NIE practice, must include -the following: 

^ Little or no advance information about specif ic competitions 
^ is available except through personal contacts, which effect- 
ively confines -participation to tfiose with well-developed 
private intelligence systems or those already so aggressively 
in the market as to scan the Official Federal procjirement 
publications. Specific competitions open and close on very 
tight schedules, leaving little time for informal contact to 
spreadL the word. The process of advertising contract pro- 
curements is especially cumbersome and ill-suited for reach- 
ing the academic community. » 

Nor does any general information at a decent level of speci- 
ficity exist concerning the overall" thrust and scope of each 
program, so that a person could review planned activity and 
^ inquire further how to participate. ' 

° No single document is available^such as a monthly list of 
present Competitions or RFPs available '^t^ough we recognize 
that fewer new competitions were held in ^he tight budget 
year of FY 75). 

° Mailing lists, we understand,, have been a continuing source 
of difficulty. Apparently the Institute does not have a, 
ready way to identify people who have indicated interest 
in the past. 



No general information exists about the mechanics of gran 
and contract application and award procedures, such as a 
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general guide to "llow to get funds from NIE-" Even * • 

small basic research projects are announced and awarded 
through the formidable method of the RFP and the contract^ 
without guidance as to the meaning of the various parts* 
to be completed, the full prcrcess that will be followed inu 
making decisions, rights of debriefing, and so forth. "^^X, 
perception is widespread in the field that doing business v 
x^ith NIE is 'fextremely complicated, and it is probably right. 

We understand that so inadequate is the Institute's general publica- 
tions effort, some Qtaff have been reduced to sending out xerox copies 
of the staff newsletter or the interim telephone directory for lack of 
anything else timely, cotnprehensiye,^ and :^nformative. 

To be sure there are exceptions, and we have seen isolated plans, 
concept papers, or documents from programs listing active grants and 
contracts. And a newspaper-format publication has begun to issue in 
the last year, though it focusses on isolated projects, not on the In- 
situte itself, and is not aimed at the R&D community, fyen staff mem- 
bers on the Institute's authorizing committees on Ga,p^itol Hill shake 
their heads that a place with such a need of public un'derstandihg, 
and with presumably some expertise iir communications and teaching, 
has such meager ways to inform the 'pi^ofessional community with which 
it must work to carry out its program, let alone educate the^broader 
world about It^ activity. * 

In short, we' doubt that NIE R&T) ^opportunities are as widely . 
known as they should be, if one is to put confidende in open compe- 
tition.^ And when we add in all those who now avoid NIE because of 
shabby treatment in the past (proposals lost, a4vicQ asked at hastily 
called convention conferences, red tape of grants management), and 
the others still who would be supe^^b if joined with one or another , 
across the hall or down the street, we begin to add up a large segment 
that may not be fully engaged by the present process. We are not 
surprised to hear NIE staff comment after procurements, "We were y 
surprised who bid and who didn't," or "We just didn't get the quality 
of proposals we hoped for, but we had to go ahead...." 

The impression left in the 'field, which we tend to share, is that 
reliance on open, na^onal competitions under the procedures now used 
by the Institute can be in part a rationalization for^being ill-informed 
about present performers and their capacity to continue present work . . 

or redirect themselves to new areas and for avoiding the substantial 
task^f identifying and being in touch with the diverse segments of the 
R&D system that should be brought together to play a part in NIE's work. 
We recognize the dangers of simply allowing each program manager to 
award funds non-competitively , and we agree that review processes are 
useful and necessary. But the coasts and inadequacies of the present 
completely open competitive methods are all too plain as well, so we 
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conclude that procurement management is a prime area for ^additiortal 
staff, research, and innovation. ^ We present a number of recommen- 
dations towards that end. We are simply disheartened to see the 
weak links between the Institute and the community of R&D performers 
after three years, and the apparent reliance on wasteful mass invita- 
tions for proposals to determine who is interested and who should -be 
considered to do NIE's work. 'V 
_ 

Regional laboratories and R&D centers 

Though commonly lumped together in language and policy as "lab^ 
and centers," we f in^ them liard -to deal with that way, as they share 
little except the accident of transfer from USOE at^^the same time 
and their status aslnltiatives of the Federal government some years 
ago. The tables in Appendix C demonstrate their variety of budget 
size., staff, and degree of dependence on NIE- The narrative data 
they submitted fur_ther^ underscored the point, as we mentioned in 
Chapter III above.__ ^ . 

Laboratories seem to us a unique structure, ^poised between the 
university 5nd the iservlrie-delivery system of education for a variety 
of purposes. Their shorif comings we feel are partly the responsibility 
of the Federal government for not seeing to it that events proceeded 
differently, and partly the result of the weak quality of some of 
their staffs and-^jqards. University-based R^D centers seem to us 
only one way of organizing university talent, with certain , drawbacks , 
and we favor use of ^various options as suits the worl^ and the campus. 

So how does NIE policy seem likely to affect these organizations? 
First, what is the policy? Leaving aside the question of . how NIE will 
respond to the conflicting signals from the House and Senate regarding, 
earmarked funds, NIE policy towards laboratories and centers seems' to 
include the following elements : . , 

. V Attempt td ^complete present products and terminate 
. plans for further work by Fall 1975 when many three- 
year contracts expire. 

Where specif ic continuations are desired on specific 
projects, individual program offices will arrange to 
present sole-source award justifications for Institute 
review. 

'^'^ . • " ■ 

"""Our own analysis of procurement issues and specific alternatives that 
should be explored could not be as extensive as we wished. Others have 
been working-in this field, however. We refer to John G. Wirt, et al, 
R&D Management Methods Used by Federal Agencies (Lexington, Mass.: D.C. - 
Heath, 1975)4 Volume 2, Part B of the Report of the Commission on Govern- 
ment Procurement (Washington, D.C.: GPO, 1972); and the active research ^ 
which may be most useful of all, the National Academy of Science "Study 
Report on Social R&D." This last group is giving particular attention 
to procurement issues. ^ ^ 
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) Certain specific competitions may be designed to draw 
' excltj^ively on laboratory or center expertise^ in such 
• activities as dissemination and product installation. 

^\^pme promises are being made about help in finding 
other funds. 

Encouragement^ is offered for institutions to |fiter 
any other Institute competition in the coming m)nths . 

Lab and center heads tell us about deep staf^cuts they antici- 
pate, because they are unsure of funds after this Fall. ♦The tables 
in Appendix C show that labs project 400 of 500/ professional "staff 
now supported by NIE will have to be terminated for lack of other 
funding, and centers report a like figure of 200 out of 426. The 
questionnaires brought us pathetic examples of the grueling search 
for other work— includitiR re-norming te^ts for the Air Force and 
doing $1500 Title I evaluations. But NIE staff explain that over 20 
million dollars were -to beiBpent at labs and centers in the coming 
year even before the Hc^usefancT Senate directed a greater amount, so 
that such estimates are net: accurate reflations of how the full 
NIE policy will work out. I Still, it seems unreasonable to us that 
just a few months before s\jch possible serious impacts, labs and 
centers should lack information about what competitions will be held 
and when, or what sole-source awards they can expect, 

' ' 
We could not review each institution or program and make specific 
judgments and r^ommendations , nor were we asked to do so. Our brief 
visits showed us places of high energy and appealing prqducts, and 
qthers with more mixed report cards. Our correspondent?" agreed. We 
ended up feeling that no group of education R&D Institutions per se ^ 
deserves continuing Federal support, owing to the great diversity of 
work- and. activity present in the field. 

^ut beyond that, we are simply appalled at the elements of policy 
that are not in place, the apparent disregard of the history of ^Fed- 
eral involvement with the labs and centers, the lack of deliberate 
review X)f the total capacity of each institution to ascertain its 
suitability for a continued role in the NIE program. If, as a gener- 
al principle, no institution is owed a l^^ving, and even if a number 
of present labs or centers could be closed withcfut serious 'loss , 
nevertheless there should be some overall process of reaching deci- 
sions that can disrupt institutions and careers. The, absence of such 
a plan of procedure is indefensible historically, substantively, and 
politically, and we strongly urge the Institute to avoid letting the 
aggregate of. project-by-project decisions rule the final outcome. 

■""As a general matter, beyond the issue of evaluating lab and center 
work specifically, we find it surprising that NIE has -no general 

(continued on page 58) 
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We can understand the history \jh±ch broiight things to this point. 
In 1972, j^hen the nev institute wa^ j^'st starting and laboratories 
and centers changed their support mettife*ci\ from a single contract to a 
.series of project-specific contracts, e^rch with its own timeline, 
milestones, and responsible government b^fficial, a shuffle ensued 
so that projects at a single 1-ab or center have come to rest 'in far- 
dispersed comers of the Institute, subject to the idiosyncrasies of 
♦project monitoring that we discussed above. In fact, we discovered 
at the outset of our inquiry that NIE Kas no up-to-date ^.nforraation 
on labs and centers as whole organizations, "since NIE deala only 
with projects now." The rationale was no doubt convincing In 1972; 
the agency will grow, funds will be spread around and the laboratories 
and centers will be a diminishing part of the total group of NIE's 
R&D performers. in this view, special attention to these few places 
was not warranted. 

But of course the budget did not rise, and the labs and centers 
have remained a sizeable, if decreasing, fraction of the total Insti- 
tute budget. And despite the central role iof people and projects in 
such places, the NIE continues to manage their work with apparent in- 
difference to the institutions as entities, and relates to them still 
through individual project officers except for confrontations at the 
top level. And with the sudden possibility that has "arisen recently 
that some additional awards to labs and centers may have to be made 
in response to Congressional requirement, the Institute finds itself 
scrambling to consider what the places are good' for — considering yet 
- a^ain the familiar device of the open competition to substitute for 
mutual communication and solid knowledge, saying in effect, "You tell 
u^ what you think yoyt are good at doing." 

So our major impression of policy with respect to labs and centers 
is that there is none where it counts most — namely in thoroughly can- 
vassing their likely contributions to NIE's work. The result seems 



(continued from page 57) ^ . 

evaluation group, policy, or support function. Thus each project of- 
ficer, each new writer of an RFP, each proposal reader, to some degree 
reinvents the wheel or expresses pergonal idiosyncrasies, in the absence 
of central guidance and sustained attention to the topic of criteria and 
standards of quality. Sam Sieber has some thoughts on this in his essay 

0 in Appendix A* We note that such a missing piece of activity, such a 
gap of intellectual substance, makes it easier for critics of decisions 
such as not renewing: a certain lab or center contract to charge that 
politics or budgetary constraints are chiefly to blame, rather than to^ 
confront the possibility of bad work. It seems odd that NIE has beerv 
able' to resist taking a sustained look at these issues, in view of th^ 
incessant pressure on local scTiools and other Federal ageocies to be 

Q involved with evaluation. * 
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'likely to be the closing or changing of a number of institutions. 
While institutions with special relationships should be decreased . 
in number and should have their, missions clarified, uncoordinated 
actions are not the way to do it. 

Taking the goal of an institutional capacity such as we outlined 
in Chapter III, -a small number of very high quality national R&D 
facilities in education, we- feel when the set of such institutions ^ 
is in place, that they ought to consume no more than a third of NIE s 
budget. If we consider Hhat a decent operating size should be. at 
least $3 to $4 million' each, per year, it is plain that NIE can 
support 9nly a fraction of the present number. A complete review 
of programs, finance, and management at each institution can result ^ 
in a considered decision whether a place can contribute to the NIE 
problem areas and the 'degree ^f reorientation needed to do so. 
Only such a detailed review and holistic judgmeht can do justice to 
the Federal investment in building the institutions. We think such 
a review process'should be done soon even if NIE's budget were twice 
its pr esent- size. Where reviews indicate that a place will indee^d 
no longer receive significant NIE funds, the concept of transition 
funds should be revived, to aid in shifting to other missions and 
funding sources, or to aid in orderly and considerate closing of 
the institution. 

Where institutional capacity seems strong and appropriate to 
NIE's problem-areas, we do not advocate returning to . the USOE manage- 
ment methods or to some new variant of institutional support. We 
sense that organizations in the field do tiot expect this. We do 
sympathize with their hopes for stability of funds, but we would 
argue that institutions which seek such security must be willi^ig 
to allow substantial NIE involvement -in planning and carrying out 
the work and not expect to be given a large contract and left alone 
for years. Three-to-five-year awards should be made to the smaller 
^et of institutions, for earefully-worked-out activities oriented 
-to specific goals and ne°eds of NIE programs. Evaluation criteria, 
dissemination plans, deliverable products, must all be specified, ^ 
though with the right to redirect them as the years pass. 

Our 'recommendations include both the general goals just outlined, 
as well as some 'sketches of a process for moving towards them ^hich 
we'^hink should begin immediately. However, the need |or action on 
many>present lab and center contracts, and for spending Congression- 
al!/ earmarked FY 1976 funds, will no doubt make it impossible to 
/bnduct many institutional reviewa in timeN:o integrate decisions 
on past contracts, current FY76 one-year awards, and future special 
status We recognize that some decisions must l^e made on the quality 
and relevance of specific projects, without, such an overall context 
to place them in, and we accept that though we deplore the circum- 
stances and history which make it inevitable. 
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We believe that institutions in danger of losihg substantial ' 
funds as a result of such judgments should be candidates for the 
earliest review, not so much on the assumption that high quality 
has been overlooked completely,^ but to assure that no re-configu- 
ration or change would alter the project-by-project decisions 
made. For the rest, we would ^xpect^that criteria and procedures 
would be decided and a system in place early in 1976 so that re- 
views cAuld proceed, with the^ entire set ex^lnined closely within 
three yekrs from now, (Note that -we do not set a ta?|et of a 46f- 
inite niimber of institutions which must be in place in the new 
relatioT^ship by tKe end of the process. We expect no more than 
sijc to eight will qualify, but it in|y be less. KJixtensive dis*- 
cussions with the profession, the pubrlic, and xlongress may be 
useful along the wayi to explain the goals an d^j ^eLig r o c e s s being 

followed. ^ 

w 

And, in keeping with this schedule, we would expect the 1977*' 
NIE planning process to, include attention by the Director to pro- 
vidin^ the necessary staff and resources for such a review. And 
competitions for laboratories and centers in Fiscal 1976 could 
well forecast the future relationships we have in mind, by stating 
in advance specific NIE priorities in considerable detail, rather 
than encouraging submission of various plans and hopes that may have 
little relevance to NIE areas, ^^this way FY 1976 funds would be 
used deliberately to emphasize a transition of policy, rather than 
simply spent on hastily-designed continuations or expansions unre- 
lated to likely^uture decisions. 

Thus, we return fto our notion that NIE work towards a small- 
er set of high quality institutions with which it will work very 
closely \to carry out its missions, managing them towards goals 
the agency and the institutions, can comfortably share. In return, 
the performers receive flm assurances of long-term support. Eval- 
uation during and at the end of the time-limited charter, would de- 
termine whether the relationship should be renewed and on what 
terms. This result will not come about from piecemeal competitions 
and RFPs. It must be planned for cieliberately , and the special status 
capitalized on for the advantages it brings for both sides, not seen 
as some sort of continuing embarrassment to be voided as soon as pos- 
sible. Monthly meetings of directors; program, financial, and man- 
agement reviews every so often; perhaps an Associate Director fpr 
Special Centers; these are some of the management tools that may be 
needed. The environment now is not that of 1972, with its heady 
hopes for rapid expansion; NIE policy must reflect that change and 
must seek to build on what exists in the most careful way possible, 
not leaving to chance the major decisions about the future of a signi- 
fiCan£-svR&D resource. 
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A new role: pf^^vidlng leadership to the R&D system J . ' 

We \ recommend that the NIEy^hrough its Director, Council, 
and a new staff unit, take on/vigorous analysis and leadership tasks 
with respect to the overall epucation R&D effort in the nation. 

From our reading, visHfing-, and corresponding, we were re- 
peatedly struck by the' diversity of what is happening in the field, 
and the need for NIE to know about it, analyze it, and speak out 
fpr the role of R&D. It was the initial intention, and we think a 
very good one, that NIE" be the agency of the Federal government not 
only for supporting programs, but also for developing policy with 
respect to educational research and development. NIE does not,, and 
need not, have in its budget all the Federal government's dollars 
spent in support of education R&D. But it should have enough of the 
total amount to' be a balancer, to bring about coordination and in- 
tegration among the Federal programs. Sufficient amounts should be 
allocated to the various mission agencies because .they can do a bet- 
ter job of meeting specific agency obj^tives than NIE is able to y 
do as a general-purpose agency. Also-, allocating funds broadly can 
promote desirable interagency competition. • 

However, NIE seems at present to give practically no attention 
to the planning and coordination of the entire national effort ino 
education R&D. Of course there cannot be a tightly-controlled, top- 
down system of policy planning run by NIE. But NIE should have -suf- 
ficient staff at a sufficiently high level that it can develop both 
an informational and an analytic overview and understanding of w^at , 
is going on in the Federal government and outside , what the key issues 
and accomplishments are, what the key problems and failur^es are. Its 
rple in these respects should be intellectual and its approach fKiould 
be to lead by force of analysis and intelligence rather than to lead 
by dictating. Its influence over policy in the Federal government 
would come primarily from the depths of its knowledge and the imagi- 
nation of its proposals rather th^n from bureaucratic power. Its 
impact on research elsewhere will similarly depend on the quality 
of data, insight, and analysis, but also on aggressive publications, 
and professional contacts by credible members of NIE staff. ^ 

^ All this will require a very sophisticated research and analysis 

unit at the very top of NIE. It may also call for broader authority 
. than NIE now has, to obtain access to information and developments 
going on in other Federal agency R&D prpgrams and outside as well. 

„ to Ck • 

The present handling of several ppf'tentt.al methods for this role ^ 
leaves much to be desired. Ffrst, a small unit with the respbnsibility 
to study and monitor the R&D system is lodged in the Dissemination of- 
fice, with useful pj.ans but limited support and apparently little re- 
lation to broader policy. Second, the two statutorily-required annual 
' . reports, one to the Assistant Secretary of HEW for Education^, on the 
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current status and needs of education research, arid another to the 
President and Congress on education and education research In gen- 
eral, xTOuld seem prime opportunities for displaying such. analyses ^ 
and speaking out for how the whole might be more than its parts. 
Rather, these reports seem to be taken as onerous chores, and the 
first report to the President only recently, limped from the press 
in the form of a tedious and ill-explained list of contracts and 
grants. 

. Now that several uncertainties are behind the Institute and a 
permanent Director is in place, such a new role can seem, more ap- 
pealing than it would have been under a "zero budget." A good 
deal of excellent work exists both under NIE sponsorship and else- 
where than can be built upon, and we note again the conceptual frame- 
work presented by Sam Sieber In his appended paper. Thus it is- not 
as though the analysis would need to start absolutely from scratch 
or without guidance.. So long as the nation's R&D capacity is as 
limited as we think it is, every bit of intelligence used in think- 
ing through how to use it best will be well-s'Jent^ In the political 
arena, too, a well-thought-out position commands respect, even if 
not agreement. NIE can lead the Congress much more forcefully than 
it Has, with strong data on gaps and needs in educational research 
and development. 



VI COl^LUSIONS AND 
RECOMMENDATIONS 



We end by summarizing our conclusions and "being specific a^oiit our 
recommendations. Our conclusions are placed in three categories— 
concerning the R&D system, the NIE, and the regional laboratories ai^d 
R&D centers. The recommendations are addressed to Congress and the NIE. 
As a context for these thoughts and suggestions, we want to comment in 
general about the whole enterprise before moving to specifics about 
where-it is today and how the Federal role might be strengthened. • 

To understand the effort that the United States is makiAjS. through 
NiE to support educational research and development that will produce 
demonstrated improvements in teaching and learning, one needs a world 
perspective. Looking at the economically well-developed countries of 
the- world, one finds precious little successful, applied educational 
R&D. There are some centers in Sweden, the United Kingdom, Germany, 
Israel, and a few other places where disciplined inquiries about 
education by social and behavioral scientists are providing the basis 
for changes in educational policies and program. But the general 
picture is one in which expectations exceed performance, and measurable 
improvements in the learning and behavior or students are difficult to 
demonstrate. 

In the less-developed countries educational research and" develop- 
ment has even further limitations. The supply of trained social and 
behavioral scientists available to work on R&D is so limited that the 
first task in many countries is to prepare c'apable people. Work no^7 
under way tends to focus upon the evaluation of a few experimental 
projects or .upon* accurately describing some of the problems that exist 
prior to attempting their solution. 

Seen in this perspective the United States has a rather large and 
sophisticated commitment to educational research and development. 
Indeed, many nations of the world look to us for guidance and stimula- 
tion in the field. Yet if this is true, why do we find ourselves 
discontented or even disenchanted with the results of what we are about? 
The answer to this query is not simple, but it must certainly include 
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the following: 1) Social science research generally is 'only slowly 
developing the sophistication that allows it the luxury of predictable . 
results; 2) The problems of bringing about and measuring changes in 
human learning and behavior are vastly more complex than those of 
technological change and are cut across by difficulties of cultural 
tradition, linguistic style, and emotional factors that simply do not 
exist^'to the same degree wh^n one is dealing with things rather than 
people; and 3) The need for improvement in the results of education 
is 80 clear and so great that all interested parties (legislators, 
educational policy makers, teachers, and parents) develop an initial 
^'^^^ii^asm for educational R&D only to have its halting and limited 
results seem at the least unsatisfactory and at worst inexcusable. 

Against this background it is well to ask whether investments 
in this difficult field are worth the money. The only possible reply 
is that we must keep plugging away at the difficult problems of learning 
and teaching and that doing so by orderly scientific inquiry is almost 
certainly better than by hunch. More is known today because of educa- 
tional research about how to motivate children, about how to develop 
and try out methods and materials, and about how to measure outcomes. 
That we do not have the final answers in any of these realms is no^ a 
valid reason for abandoning the effort. For the United States, NIE is 
the central expression of that difficult and frequently frustrating 
enterprise. 

Conclusions regarding the R&D system 

1; The set of people available to do^education R&D and link its 

results to schools is too -small .for the job, at most no more than 10,000 
people altogether, and fewer than that in person-vears of effort. 

V Regiji^ial laboratories and R&D centers include over ten per cent 
of fhe total staff now doing R&Dj^ and consequently must be viewed 
as a substantial resource,. / 

2. The R&D system outside the labs and centers is fragmented and 
hard to organize owing in part to the meager nximber of R&Q - 
- groups in diversities, cbnsiiUing firms^ state education 
. agencies, local school systems^^nd other places. This 
fragmentation lays a heavy burden on NIE to reach out^in 
creative ways to bring the scarce and di^ociated resources 
together to ^ork effectively. 




While basic research must continue, and needs support wlierever 
It can be found, an even more pressing need is for both concrete 
program development and readily available aid to local schools 
in using the results of program development done elsewhere.^ 
Merging the functions of research, product and program develop- 
ment, and service to schools in institutions such as labs and 
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centers may cause inefficient use of available energies, -and we 
stress clarification and even possibly a more specialized set 
of functions at least for some national laboratories. 

4. • Though we consider there to be a need to protect both^ university 

researchers and the renev7ed national laboratories from undue 
pressure to serve schools directly, there remains then a gap of 
technical assistance atid support for schools which are trying 
to imprcrvfi. Current approaches conceptualize the issue either 
in mechanical terms of how to /'disseminate" ^rom a national 
R&D bank, pr exclusively in terms of local self-help. The 
former notion is full of substance (products, research findings, 
teacher guides, exemplary practices)' but lacks any motivational 
basis. The latter is content-free and imbued with social- 
psychological assumptions about the nature o^ innovation and 
self-renewal . Our conclusion is that neither strategy alone 
-A will make much difference to education, and that they must be 
^ Integrated. We are discouraged at the predominance of discip^ 

in much of the field and in key parts of the Executive Branclr 
: ' outside NIE simply in terms of the mechanical model of 
.dissemination. * 

5. Each of the system,'^ institutions has its own sense of function 
and interests, and with few exceptions, inquiry into mission pri- 

"Orities is not one of them,yfior is deliberate change in response 
to the results of inquiry-jpne' s own or someone else's. It is 
thus extraordinarily diff icull: to involve people in an R&D agenda- 
setting process in the first .place, and harcier still to attain 
credibility for the more searching questions that might be asked 
about present educational arrangements. NIE should receive credit 
for attemptfrt^^o build some consensus around certain areas of 

. focus for its R&D work. 

- 6. A variety of persons and divisions in universities, and in state 
and local education agencies, are underused or even ignored as 
parts of the national R&D effort. Minority persons^ and women 
are especially underused in the overall R&D effort. 

' 7. We may be seeing a trend in the world of education R&D towards 
emphasis pn marketing, the fierce rush to attract new business 
' for R&D performers, and diminution of emphasis on perfprmance— ^ 
in research, product quality, or attention to ultimate needs of 
schools. This legacy of shrinking funds — the resulting scramble 
by R&D institutions to maintain themselves — is not likely to 
change in the next few years, but will have pemicio^as^ consequence^ 
if it does persist. 
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Conclusions regarding the NIE 

8. The staff we have met seem able, perhaps above the average 
caliber of * government; staff, and they seem to be working 
remarkably productively in face of many difficult circumstances 
We conclude that despite recent Civil Service criticisms and 
resulting uncertainties about future personnel policies, the 
Director and Council will need vigorously to defend NIEls 
partial exemption from civil service recruitment and hiring 
practices. Many more top-level research managers and experi- 
enced practitioners will be needed in the next few years, 

and forced hiring from within government ranks will make that 
very much more difficult; 

9. Staff appear to have diverse perceptions of the major role or 
objectives of their agency. By this diversity, we mean that 
program staff appear almost without guidance as to how to relat< 
to individual pieces o^'work under way through , grant and 
contract, how deeply to be familiar with them, how much to call 
on others for help in monitoring particular aspects, how much 
to intervene, how much to be an advocate or critic. We feel 
such diversity has caused great difficulty for NIE in the field 
owing to inconsistencies as project officers change. 

10. The haphazard policy of monitoring, and the dictum that all 
new funds be throra to opai competition, seem likely to combine 
to reduce the incentive for performance of quality work by 
contractors and grantees, since outstanding performance is 
recognized only in hie or miss fashion.^ .-^^i^ note that in 
several other R&D agencies, con tinua^r^oir^^r ants form a 
substantial fraction of the awards, anjf teview of such 
continuations prbvl'dtes at least some /^^uality control. (Though 
considerable funds were given out ia/ continuation" awards in 
recent years, they were in fact iacremental funding decisip^s, 
not t^e reviews and fresh decjUions to conti'hue -a line of work 
buil^lrag^on the past.) / 

11. Diversity of approach to monitoring and review By project 
officers result in part from the past NIE policy of ignoring 
labs and centers as coherent entities, and relating only to 
projects. We find such a. policy* confining and insupportable 
in the present context and for the near term. It results also 
from almost total lack of agency-wide attention to issues of 
evaluation and quality standards which we find hard to justify. 
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12. While there appear to be excellent relations in many cases between 
project officers and projects, based on idiosyncratic mutual 
adjustments in each case, there is strong distrust and miscommuni- ^ 
cation between the field and NIE at higher levels, since decisions^ 
such as genera^ funding for types of activity appear to be made with 
little or no knowledge about institutions or projects. 

13. We are disappointed not only at the inconsistent amounts of 
information available — and the lack of policy governing such^ 
knowledge — at the project and institutional. levA, but, at the 
level of^ components of the R&D system as well. We conclude 
NIE has not yet taken on its rightful role as intellectual 
leader of education R&D, both within and outside the government 
setting forth directions and analyses on the subject b^sed on 
wide knowledge 'and top-quality analysis. 

14. For giving direction to its own funding and management, we 
conclude that t^e new NIE program structure centered on problems 
is prudent in the short-term, though we hope that in time both, 
basic studies and internal research can gain prominence along 
with extramural work directed towards solving specific problems, 

15. The process of choosing sub-priorities and strategies for 
attaining the goals of each problem are^ does not reflect as 
much care for broad patticipation and building legitimacy, 
nor for taking into account the work presently under way, 

as did the initial choice of problem areas, at least in the 

eyes of the field. 

» * ■ - 

16. We conclude that poor corfiunication with the public and potential 
performers of NIE's leerfc renders less meaningful the idea that 
open competition is a useful strategy for finding the best possible 
performance. Little imagination seems to have gone into linking 
NIE R&D(Jneeds with potential performers through creative adaptations 

, of the typical procurement nethods used for nuts and bolts. 

17. In particular, little attention has be^.paid to bringing the 
local schools and state agencies into partnership with NIE as^ 
agents of R&D, not simply passive recipients at the last stage 
of an R&D process. Attempts to improve states' ability to 
disseminate products and knowledge produced elsewhere are an 
important effort, but to some degree perpetuate notions of undue 

' functional distinctions among agencies and a mechanical dissemina- 
tion process that we feel do not represent reality, and should not 
be enshrined in NIE policy. We do not mean to imply a conclusion 
that no help be given to states to d:evelop an extension or 
dissemination system, only that state and local agencies' capacities 
fpr engaging in other functions of R&D need to be equally develoi^ed. 
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18. Iilhile v;e cannot claim to have made a thorough review of the 
relationship betv;een NIE and the Department of HEW, signals 

from a number of sources suggest that there is room for improvement. 
The considerable emphasis placed on making NIE independent of the 
Office of Edufcation may have had the effect of giving it an 
unintended distance from the Secretary's office in HEW. There are 
indications that NIE has not received supportive leadership in 
formulation of its policies and in gaining Congressional acceptance 
of its budgets and appropriations. While NIE is a small agency in 
HEW in budgetary terms, it has a povjerful potential for the educa- 
tion system of the entire nation — an enterprise on x^^hich close to 
.$100 billion is expended annually. We conclude, therefore, that 
both NIE and the Secretary of HEW need to jgive further attention 
to developing a relationship that will advance the mission of NIE. 

19. Despite the continuing contrbvers^r abput the value of currlcultmi 
reform as a strategy of change -in education, vre see little clear 
policy analysis at^NIE on the subject. . 

20. We understand the political pressure for "dissemination" of the 
results of R&D, but we conclude that NIE has done little to attack 
the problem as a substdtitive matter or cluster of issues and 
competing^ conceptualizations* We do not think that work in the 
field can be halted until theory catches up, but we do believe 

an experimental attitude would be helpful even as adtion goes 
forward, and that diverse groups x^ithin NIE could be brought 
together more directly to consider paradigms for change and the 
various roles of "diss4mination" within them. Research on know- 
ledge-utilization could be more extensively funded as an essential 
basis for policy in this area. 

Conclusions regarding the regional laboratories and R&D centers 

21. Failure of the laboratories to reach some goals held for them at 
the outset seems to us chiefly a failure of the government to guide 
and encourage them towards those goals, not a failure of the concept. 
The cpncept of a specialized, separate agency in touch with schools 
but able to retreat from direct service to test ideas and devel.op 
new programs still seems distinctive and sound and worthy of 
extensive support. 

22. ^ Actual performance ^^y the laboratories and R&D centers appears 

to be widely varied lly any standard, and judgment is complicated 
by the variety of activities within and across institutions, as 
well as by the complex set of claims and counter-claims about x^hat 
each lab or center has in the past been directed to do. Regional 
service, contributions to knowledge, proven product impact or wide 
dissemination, all Compete for preeminence in various eyes. We are 
^ concerned lest hasty review of --product impact, for instance, be 
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used as the exclusive measure of a laboratory's present value or 
^ future capcity.^ It| was not possible for the consultant group to 
develop and apply a detailed set of criteria, aild we- believe the 
NIE staff should do that work in any case. .Considerable work 
will need to be done, judging from our discussions with staff, 
to arrive at criteria of quality, and relevance that can be 
shared. ' ^ 
A 

23. Despite this difficulty pf assessment, we find at least strong 
rhetorical agreement among this set of institutions that they 
do not expect "institutional support'' unrelated to performance 
and to tA\k relevance of their work and that they would welcome 
straightforward and tough-minded oversight by consistent and 
qualified government officials and. outsiders. 

24. We conclude that the persisting dilemma for. Federal policy towards 
the laboratories^ and centers has been the large number of institu-". 
tions to which the government seemed commlTtted, and their very 
mixed quality of work. "Since the first few years of each program, 
efforts have been made to both reduce the niflnbers and encourage 
higher quality. We conclude that there may haVe been more success 
in eliminating marginal institutions, or at least ending their 
substantial Federal support, than success in improving the quality 
of work at' those remaining. We find it essential that both trends 

of policy continue, to result in a still smaller number of institu- , 
tions dependent for major support on the NIE, with improved qtxalitv ' 
and relevance of effort, and related to the IJIE in special and novel 
ways. 

25. Moving from the present situation to the one we envision — of more 
diverse ties to universities, and a smaller set of national 
educational R&D institutions— will not be accomplished effectively 
by allowing individual project contracts to lapse and beginning 
new work through piecemeal sole-source awards or open competitions 
in whicha, labs and centers are simply free to enter along with 
others. '' Instead, we conclude that NIE must .give priority in 
planning and procurement to dealing holistically with eaph of the 
present set of institutions, with the goal, of pri^serving and 
strengthening a smaller number with significant potential. 

26. We expect that no more than six to eight of the present laboratories 
and centers would meet comprehensive criteria of quality and 
relevance to NIE's missions, upon close examination of entire 
institutions. (Other estimates we have received are even lower.) 

We do not resolve upon any pirticular target figure for the nuniber 
of' institutions that should emerge from such a review. Those 
which do meet criteria of overall quality and relevance will need 
to be substantially funded to provide a necessary "critical mass" 
of R&D activity. • 
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27. After a deliberate process of review of all aspects of each 
present lab or center, and consideration of redirection or 
reorientation in each case, it may then be appropriate to 
consider new candidates for a special relationship with the 
NIE. We conclude that extensive new competitions seeking 
such candidates are unwise, and that bringing existing f 
non^-lab^-or-center organizations in,to such a special status, 
or, encouraging the establishment of new places^ must be'done 
with great care. 

\ 

28. We conclude that in any case, no more than about a third of 
NIE's program funds be allocated to work at the resulting 
special institutions. 

Reconmendations for the Congcess 



1. That NIE be reauthorized on a long term or permanent, and 
with authorization of appropriations substantially above 

the $80 million per year -In the administration bill. * ' 

2. That explicit authority be given the NIE, if necessary, l^o allow 
it to collect data from other Federal agencies concerting 
their education research and development activities, and from 
organizations outside the Federal government as well. 

3. That the new legislation on the Institute continue, the concept 
of a strong policy-mking National Council, including terms 
longer than the present three years. 

4. That the new legislation continue the authority for the 
Institute to staff its activities ^flexibly, to allow for 
bringing in the uniquely ^alented individuals that are needed. 

Reco.mmendations for NIE 

5» That the present problem-foci be retained but that, these be 
strengthened by establishing a fotmal advisory bo4y for each 
program which could link the Institute with various constituen- 
cies^ for its work in each area, aid in reviewing specific 
awards thus freeing staff to consult with proposers, and serve 
as a sounding 'board to insure that the "definition of the ^ 
problem" remains current. 

6. That much detailed thought be given to the role of prbject 

monitor and how NIE wishes monitoring to be performed. Such t^sk 
force thinking sliould be^ followed by appropriate written policies 
and staff training. In the course of developing such approaches 
^ we urge the study of practice in other R&D agencies and 
consultation and surveying of present grantees and contractors. 
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That the Institute establish an "R&D Syfetem Monitoring and 
Analysis Unit*' at the highest level, which can have staff and 
resources to take active part in the Director '.s decision-making 
and planning, and in the required reports to the Department, 
the President, Congress, and the public. The 1975 Databook is 



unit. We would hope that the Nrfechnical experience of • gathering 
data for the volume might be the basis for a national meeting 
on sources of data on education R&D and how to strengthen the 
collection and reporting of necessary indicators, with advice 
sought both from the field and from other/agencies such as 
NSF with wide system-monitoring responsibilities and experience. 

That NIE completely review and revise its efforts to inform 
the R&D system about NIE. We urge^the mandatory pubW-cation 
by prograin offices of their plans and procurement expectations, 
^d names of people who can be contacted for further specific 
information. ' If mailing lists continue to pose/problems for 
those who do wish to communicate with^vario^ groups we hope 
outside help Will be sought to devise a modem and effective * 
system. We make other recommendations concerning publications 
under the recommendations on procurement and state and local 

agencies below. 

.1 ¥ 

That the NIE deyote explicit and public attention to issues of 
the present/ value of curriculum^development as a strategy for 
akiing schools! We sense strongly - held and diverse views on 
the subject, and there appear to be policies made, in part, 
on private judgments of the merits of the case. But we notice 
little public airing of Jhe debate. Some writingl convening, 
and general discussion of the value of Federal support of 
curriculum development jwould be a national service, as well as 
useful in reaching and explaining a key NIE policy choice. 

\That since the goal of quality work requires a diverse array* of 
pyocurement approaches suited to each type of R&D work, the state 
of the community of performers, and the preferred role of the 
NIE staff, NIE should devote resources in the near future 
to strengthening its repertoire of available tools and the 
abilities of program managers to use chem. We recommend, for 
example: 

Review of the contracts and grants office staff size, and 
the adequacy of their backgrounds, for insuring timely 
and relevant advice to R&D program managers on procurement 
and business management techniques suited to the special 
citcumstances of ^&D. 
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Closer working relations between contracts staff and 
program staff, from the inception of design of a procurement 
80 that plans can take into accQunt the substance of what is 
wanted, how to> reach those .who should know of it, and a 
range of possible ways to conduct the procurement and award 
the funds. More formal circulation of "good" announcements 
and RFPs,' with key points highlighted by contracts and 
other staff, would be educational, also. 

Review of R&D procurement procedures used elsewhere in the 
Department, as well as in AEC, NASA, and NSF to gain a 
broader sense of options available. Among the possibilities 
that should be explored are: 

• — greater use of restricted competitions, such as by 

region or qualification; 
— ' use of review processes which allow program staff 

to work with proposers at an early stage; 

— based on iden,tif iuation of resources at a university, 
use of block grants to a dean or other individual with 
discretion as to how to fund and organise the resources; 

— use of continuous competitions in areas that have 
continuing interest to the NIE,.with open dates and 
deadlines publically announced long in advance and 
awards made periodically; 

— funded design competitions. 

Exploration in the near future of ^ the potential of several 
new publications relating to procurement: ^ 

\ 

— a general guide to obtaining NIE funds; 

— explanations and instructions covering all parts pf 
grant and contract competitions, including certifications) 
competitive range, clarification question phase, 
negotiations* and award, including notice of Institute^ 
commitment to debriefing; 

- — monthly or other regular listing of competitions in 
process and contemplated. 

Establishment of a procedure for recordijlg judgments of 
past work by grantees and contractors, ^nd procedures for 
using that information fairly in future^ompetitions or 
other procurement actions. 
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^ Exploration of the possj^bility of regloijal seminars, ^such as 

those held by NSF's RANN (Research Applied to National Needs) * 
Program, to acquaint people with research directions and 
application details, for a number of NIE {Programs at once. 

Complex questions of lav; and policy- arise in some of these 
suggestions, and we imagine a substantial study effort will be 
needed over the next year 5 perhaps including several invitational 
conferences of people with wide familiarity with R&D management 
tQr consider the state of the art of R&D procurement in 'education 
and its impact on the R&D system. Continued attention will be 
needed after development of better tools of procurement, to 
insure that program staff are aided in using them fully. Few 
other tasks are more important. 

i 

11. In program planning and in procurement, special attention must 
be paid to overcoming the assumption that schools, school 
districts, and state agencies are merely the recipients of 
the fruits of R&D. We' suggest as one possibility that NIE 
establish a^i state-and-local-agency liaison unit, designed to 
monitor R&D being done by these organizations, to link them 
together in useful ways, and to draw their attention to NIE 
activities of special interest. All procurements should be 
monitored by this unit, to avoid embarrassments such as in 
the past when states or local agencies were inadvertently 

left off eligible lists. The unit could perhaps be part of the 
R&D System Monitoring and Analysis office we recommended^ 
above. In addition, the unit could* help these agencies apply * 
or compete when( appropriable. We hope NIE will consider a 
special publica5tion,,,fo3?^ocal school districts and state 
agencies, desig^d to make them more aware of R&D generally 
and the NIE role as well. Contracts with other orgajiizations 
, should have clauses which encourage them to be sensitive to 
the concerns of teachers and administrators in all phases of 
their work or which could even go so far as to require 
participation of local schools' staffs at key points in R&D. 

12. Though we agree with the overall direction towards ^problem- 
focus, we urge the Director to reserve a modest sum for 
special opportunities which fall outside the problem structure, 
and which may suggest new problems needing attention. (We 
comment below on the need for a general long-range planning 
and problem-sensing process.) ' 

r 



^76- 



13. That in connection with the R&D System Monitorlnig and Analysis 
unit we recommend, the Director consider helping the Council • , ' 
establish a special advisery group to itself on the R&D 

system. This group of experts on science and R&D could 
review the "map" generated by the staff, and comment *on It 
for the Council in reports from time to time. Such a double 
input would give the Council both expert staff work and 
analysis, and also some reflection on its meaning for policy 
by a small group which -could build up substantial insight if 
allowed and encouraged- to persist over several years. 

14. Because-we are concerned not to lose thfe tie between education- 
related research and the larger scholarly conversation,, we urge 
that NIE make special efforts to inform diverse elements of the 
university and college faculties of its programs and opportunities. 
That NIE find ways in procurements to allow the vital brokering 

of diverse interests that would not come together without' an 
outside vision. That NIE maintain contact with such general 
scholarly associations sls the Social Science Research Council, 
American Council of Learned Societies,, and American Association * - 
for the Advancement of Science^. One suggestion that we reiterate 
in this connection, is the idea of avj^rding block grants to deans 
or other university officers to support ^wbrk by younger faculty, 
or to carry out a specific set of activities. " 

15. (a) Because of our conclusion about the under-supply of 

education R&D personnel, we have given some thought to - 
methods for increasing it. We do not have a plan to 
recommend, and we recognize that hoth tight budgets 
administration policy argue against some. of our ideas. 
But we commend these scattered notions to the Director 
and Council for further study. We think that eveii 5% 
* of the NIE budget could be A very useful investment 

at this point, if spent in some of the following ways: 

Awards and incentives to individuals from a 
variety of disciplines, to pursue graduate 
study leading to a doctorate related to 
education research; .students to attend wherever 
they wish, and to finance part of the cost 
themselves ; - / 

To convert recent graduates in other fields, 
a program of post- doctoral fellowships in 
education research; ^ 

Mid-career a\i^ards to allow inature investigators 
to eas€^ into a new field without loss of job; 
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. Apprenticeship opportunities to be required in the 

ternts and conditions of all NIE grants and contracts; 

'\) -. ■ ^ 

Awards for graduate study for staff in state and local 
education agencies, on condition they plan to 
return to their home base for a period after receiving 
the degree in a research-related area. 

(b) .because of. our^cpnclufeion concerning the under-use of 
minority persons auSd women -in the national Rfi^D e^lort, 
we recommend -that NIE strengthen its pQ;(.icie8' :^n support , 
of increasing the participation of these groups in R&D. 
Such policies could include: . / ^ 

Special reviews\<^t each procuretaent to note \, 
opportunities for minority firms, and better * 
maintenance Of lists of *qualif ied firms for 
use of programs * '^^ / " ♦ 

^ . • • • - ^/^ - ■ ^ ^ 

• Aggressive review of hiring by contractors to 
insure compliance with equal opportunity and 
affirmative action law ,^nd regulation; 
*■ , '"^^ " * 

Special attention to expanding training and 
' apprenticeship opportunities for minorities > 

and women in all NIE grants and contracts, 

16. Because we senffe that attracting the very best staff may depend 
on offering opportunities for research as well as for monitoring 
tHe work of others and because research' management requires famil- 
iarity with the state of the art in the field, we recoriimend a^ 
.modestplanning effort to inquire how intramural research activity 
can be made a part of the normal expectation for NIE staff. We 
recognize that serious in-house research must await higher fund- 
ing levels, which will only come as extramural work shows credible 
results and growing constituencies. However, we strongly urge ' 
retentibn of the NIE Fellows Program as a start, and some for- 
ward planning about the shape of an eventual internal research ^ 
activity. . , 

17 ♦ Again we make a recdmmendation which must await higher funding, 
but which we feel strongly about. Most research in higher 
education is done elsewhere than NIE, but even in a limited 
way NIE could make a contribution now as a prelude to later, 
more extensive efforts. We have in mind the area of "higher 
education finance as a start. , We would hope that eventually 
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NIE might support R&D pertaining to a variety of post- .....^^ 
secondary institutions, including those outside formal 
^ , schools such as muBeums, parks, zoos, libraries, and learning 
''networks. . 

18/ Though we understand the need for more short-term-payoff work 
\just now, we recommend that NIE program areas not reject , 
longer, more complex "development" activity without cAreful 
analysis. We have in mind a number of neglected areas of 
the education program of the country, any one of which could 
profit from some clear thinking and program development 
and trial. NIE needs, and the needs of schools and teacljer^ 
may in part conflict here, and xve would be disturbed to see 
too easily written off the whole idea of development, 

19. That the NIE establish an internal task force at the very leasts 
if' not an ongoing external study and advisory group, to attadk 

.the substance of the term "dissemination," We sense diverse 
conceptualizations within the NIE and insufficient "effort to 
reach syUjIhesis, or even clarif ic-ation of complementary features 
or differences. Thinking in other /quarters of the R&D system 
on the subject is not well-advanced either, and could be immensely 
stimulated and enlarged by some NIE writing and convening on the 
- subject. We urge, that internal planning at NIE for its own 
'program consider how the "dissemination" and "problem-sorlving" 
approaches will merge or be integrated at the operating level 
. " of the school, so "that both materials and ideas, as well as t*ej5 
motivation .to use them come together for sustained improvement. 
We have noted the strong potential of "extension agents" ^ 
for such a role, and we urge that efforts to promote a major 
program of support in each state be augmented. In j^articular, 
the possibility oIe pursuing a "top-down" strategy la creating 
a national extension system should not be ruled out. The 
present "bottom-up "t^ strategy (of building on what exists) is 
useful, but might result in a weak system without common 
standards or wide visibility and support, ^ ^ 

20. That NIE establish a long-range planning process (in addMion 
to the process for choosing the next increments df wbrk .within 
the present "^^roblem areasV We recognize that NIE i? not 
complete^ it ovm master, and that of cours^^ent^ t^^rlL.pr^ss ^ 
ne^ issues upon. the Institute regardless of the bee^ prior \ • 
planning. But x^e see tio present deliberate activity aime^ at 
sensing the state ofy^ education generally to maintain a timely 
list of candidates for NIE attervtion-7nothing other\than 

' atomistic ^nd informal actions of the Director,, tl^ s^taff, and 
^ the Council, Some close work among tl^^Assistanr Secretary s ^ 
office, the National Center for Education Statistics,, and . 
Congressional ^^;goOTnl'ttees would be a start, with ^ecfiaps joint ^ 
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collaboration on some indicators that would serve to alert Legis- 
lators, the USOE, and NIE to needs to be addressed through their 
respective methods. 

21, The following series of related points constitute our thinking 
about a course of action towards regional educational labora- 
tories and R&D centers. ■ ^ ^ 

(a) We recommend ttiat NIE adopt a long-range goal of 
assuming the majority support of a small number 
of large, high-quality MT> organisations, with 
x^rhom it will have a relationship different .from 
that with other grantees and contractor^, 

(b) The national education R&D institutions which are 
the goal of the above recommendation should share 
certain features: 

.'^ Emphasis on a single mlssioji, closely tied 
to one of NIE's national R&D priority areas; 

° A purpose of following ideas from inception 
\ to utilization, with specific tasks along > 

\ the way firmly agreed to in advance; 

** Stable fynding for three to five years, at 
a level of at least $3 to ^^$4 million per year; 

° Funding chiefly' from a single source— the NIE, . ^ 

with other funds subject to review and possible 
^ . limitat^n to maintain essential missiOn-f ocusj 

Protection from demands to give local services 
unrelated to field activity that is part of the 
R&D mission; 

° Close ties to tfie major sponsor~NIE, for review 
of the entire institudtion at intervals during and 
at the close of the contract term, including review 
of finances and management, as well as programo 

(c) We recommend, V therefore, that NIE review an3 revise all 
present policies that C9ntribute to. the present situation, 
where it irffeubstantially supporting a relatively large 

r number of special- inai^titutions, of 'diverse quality, with 
. varying lengths' of i^ontract term, sfubject to uncoordinated 
NIE management and review, and inconsistently related to 
" < NIE priorities. 
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(d) We recommend t^t present regional laboratories and R&D 
centers be considered candidates for this new relationship 
to the national R&D effort and redefined mission and 
obligations. ^ * _ 

(e) We recommend that all procedures for dealing with the' 
laboratories and centers for the next few months be 
immediately reviewed to insure that actions now under 

yay are not in conflict with the comprehensive policy to 

outlined below. We recognize there^may be cases where 
the institutional ireview procedure to be spelled out 
cannot be completely\ followed, and action must proceed. 
But we have in mind ogXtlcularly not following through 
on any plaixjied compeMtions or other funding actions 
for labs ^nd centers that do not relate directly to NIE 
problem areas ^ and which do not lead towards a definitive 
policy to be followed in future years, 

(f) After estimating the available funds for th^ special 
institution^' according to guidelines of ^how much in 
^total knd how much each^^IS staff should begin a. 
^systenStic review of each lab' and center according to 
^a numb|r ot criteria: • 

. , Relevance of work to NIE priority areas 

Quality of work j^r' 

Quality of management, both programmatic ancT f in§n^^l . 

^ . ^ Commitment to utlllzatioa of results 

- . ^ Perceptiveness of how to keep big R&D targeted .to ^ 
help schools 

^ , ^ Willingness td accept- the obligations of7the new 

institiitional form being proposed * ! 

' ' ' " . ■ ■ , " • ■ 

As we have stafee^^,^ considerable Xi7prk must be don^^ to*' 
. iTeflne such a list/ wake it operational, and set standards * 
in each "area.* The^ reviews should take into accotint 
potential to contribute, not simply past work alone, as we ^ 
strongly favor the redirection of capacity before its , , 
'i ' abandonment. Basfed on the available- "fund a, ^up to the 
tvarget celling, some of the highest quality and most 
relevant institutions should be supported under the 
* " conditions^ .wfe""* will specify; others would not,' but would of 

course be eligible? to compete in other NIE activtie^, 

^ ' ■ ^ > ' 

(g) An independent external panel of R&D practitipnera, school 
and university leaders, and members of the public, should be 
convened t"© sacamine the institutional r.eview process 4nd the 
result ingx judgments and proposed actions, and report their 

'^conclaaions to the Director and the council- ^ * 
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■I 

The Institutions selected would receive the type of mission- 
specific contract for thref to five years that we have 
discussed. Extensive and complex work In a particular NIE 
problem area would be ^expected, with timelines, program 
management, and expected results so far as possible 
. specified at the start. The support would not be 
Institutional, In the sense that the organization could 
decide how It. would be spent. Beyond the development of 
the Initial detailed contract, NIE wc^uld reserve the right 
to Intervene strongly In program and management if work 
falters or priorities change. In return for extraordinary 
continuity and magnitude of funding, the special^ Instlt^itlon 
would agree to these close ties wlhih Its sponsbring agency, 
and to reexamination of the role of Its policy board. 

Applying the above^rocedure Inmedlately to all labs and 
centers Is not feasible because of the limited time 
before most lab/center contracts expire. Thus, only a ^ 
small number of Institutions can receive Immediately the 
detailed review we recommeiid. The criterion for choosing 
places to Include In the first set for comprehensive review, 

? Is the NIE staff estimate that they are xmllkely to be 
successful In finding other fxmds at NIE, on groimds of 
quality and/or relevance. ^ Other places with more substantial ^ 
prospects can safely be postponed, bu| reviews must be done 
soon of Institutions where piecemeal decision processes seem 

" likely to result In serious loss of Federal support. 

However, Immediately following completion of reviews of 
Institutions In jeopardy, NIE should begl^ reviews of all 
laboratorled ^nd centers In the same fashion as outlined In 
(f) above. These reviews could be spread over several years. 

4ife recommend that NIE not hesitate to end any sense of 
general obligation to labs or centers which, upon thorough 

<v Institutional and programmatic review, seem not to have the 
quality of work or relevaace of work that NIE needs, or the 
likelihood of attaining It upon redirection. No Institution 
should be- considered for inclusion in the special category 
we have been describing simply on the basis of its own track 
record and desire to .continue doing "the same. Such a 
special relation is reserved for highly f ocussed work of 
direct relevance to NIE. Other funding mechanisms may be 

« available to allow more self-directed work. The consequence 
of this recommendation may mean that some institutions 
actually close, or must reorient ^their work away from R&D 
in areas supported by NIE. We are prepared to accept that. 
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(1) We sense that the concept of "reglonality" in the original 
legislation regarding laboratories was added for political 

• convenience, and we recall the contrary concept of "national" 
laboratory described by the task force which initiated the 
notion. Accordingly we recommend to NIE& that some care be 
taken to maintain diversity of location of the remaining 
special institutions, so that a spectrum of potential 
problem-^foci Is noticed and can be placed as candidates on 
the national R&D agendla through informal Interaction of NIE 
and institution staff and directors. But we urge that NIE- 
avoid continuing or establishing any special institutioTi 
simply so that all areas of the country have one nearby. ^ 
We recogniSe that the closing of sizable installations will 
be a difficult task. But those places that achieve the ^ 
special support status we have in mind should dd so on the 
nationally- judged quality and relevance of their work. 

(m) Where institutions will lose substantial Federal funds over 
a short jperiod, we recommend transition funds be awarded to 
' aid in reconsidering of goals, seeking new business, and 
i:;eorienting staff. 

(n) Evaluation criteria far the special institutions should be 
agreed on in advance, so, far as possible, to guide interim 
and end-of contract reviews. It should of course be made 
clear at , the outset that the institution has a mission- 
related life-expectancy, not an indefinite, tie to the sponsor. 

(o) Since time and careful thought are needed to desigi^nd ' 

• implement the extensive review process we have in nuthd, i?^ 
we/Tecommend that the Director review the present organi- 
zation of NIE for its adequacy to the task. Since there 
now is neither structure nor incentive that brings Jieople 
from various program areas together to consider a particular 
institution, we think the Director should consider establish- 
ing a p6sltion'*to do thfs, such as an Associate Director, for 
Special Institutions. A critical first task for this 
person would be to review the procedures, criteria, and ^ 
result of the several past reviews of laboratories and ^ \ 
cetiters including those by Frank Chase, Frank Westhelmer, 
USOE, and the 1^72 panels. This experience must be built 
upon wherever poasible. • 
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This paper was prepared as an addendum to "R&D Funding Policies of 
the National Institute of Education: Review and Recpmmendatlons," 
a report by ten consultants to N^E. Although a member of this group 
of consultants, the author desired to make a more personal state- 
ment than provided for within the framework of the report. 




The Requirements of a National Educational R&D System 

- ft Sam D. Siebfer 

NIE Consultant 
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As ^ group the consultants on alternative funding policies of NIE has 
appropriately centered its attention on issues of immediate concern to 
NIE and the R&D community, to the political and economic context of 
those issues, and to concrete recommendations for dealing with them. 
But lurking behind the discussion of current problems, and indeed only 
partially apprehended by the field itself, are certain assumptions about 
what constitutes a "national R&D system" in education. To my knowledge, 
these assumptions have never been spelled out, and it is the purpose of 
this addendum to express my personal view as to what these might be. 
This task, I feel, is a vital one. 

A national R&D system, we are told, is something that NIE is 
mandated by Congress to nourish and bring to fruition, something 
which the group of consultants hopes will become more viable by 
virtue of their efforts, something which nearly all of us endorse — 
and yet, no one seems inclined to define this system. Clearly,* it ^ 
is not something that exists in the natural order of things, but 
something that we would *l1fke to approximate more closely than at 
present. These considerations, have led me to ask: What might be 
the design requirements of an R&D system in education? These retfuire- 
ments, or "functional needs" to use another set of jargon, need to be 
laid out in a clear-cut fashion so that one call tell where we are and 
where we want to- go — or at the very least, to stimulate debate over 
the underlying dimensions of an R&D system. Also, it might then become 
possible to communicate more effectively to Congress the fundamental 
ideals of such a system and the steps which are being taken to achieve 
them. 1 ' " . 

Note that I am addressing myself to systemic requirements and 
not to the ultimate intended outcomes of the system, such as equity, 
preparation for work, enlightened citizenship, etc. To a large extent, 
system requirements can be spelled out independently of the desired 
outcomes. It is like building an atomic ^conversion plant: you can 
warm homes or you can make a bomb with it. The basic design requipen,. 
ments are the same. Without a concise list of such requirements, it 
is hard to imagine how we can generate criteria of accomplishment or 
check-points for policy making and implementation. 
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virtually, the only system requirements that have been recognized 
by NIE-USOE over the past fifteen years have been functional specializa- 
tion (research, development, dissemination, training, etc.) and to a 
much lesser degree quality control . (lb repeat. Impact on practice Is 
not a system requirement strictly speaking, but a desired outcome of a 
system. It Is discussed under "goal setting.") Thus, structures like 
R&D centers, labs, ERIC, and Title IV training programs were established 
to foster each of these functions. Since other system requirements wer^ 
n9t explicitly envisioned and promoted, these arrangements led to a 
variety of unanticipated consequences, some of which were quite un- 
desirable, for example: overproduction of academic researchers, meager 
attention to practitioners' Interests and capacities for change, shift- 
ing emphases among functions (research, then development, and now 
dissemination), overlapping of functions within ostensibly specialized 
agencies, alienation of social scientists, and Inattention to substan- 
tive goals. In short, narrow emphasis on the single design requirement 
of functional specialization has ^Is^n a grave mistake. 



To be sure, functlonaP specialization Is a basic property of any 
engineered system. But the difference between a smoothly running 
engine and a miscellaneous pile of engine p£|rts Is that the former 
meets a number of requirements in addition to functional specializa- 
tion. Indeed, this particular requirement is adequately fulfilled by 
a j unk yard . ' , 

Eventually it might be possible for NIE to generate a two dimen- 
sional matrix which ^hows (1) design' requirements, and . (2) the mech- 
anisms which are intended to meet each requirement. Then, by monitor- 
Ing^the " extent t^o^which' each mechanism is doing its job, the matrix 
might serve a^^ kind of score board for NIE activities. 

I will now turn to the design requirements that I believe apply 
most critically to a national educational R&D system. Admittedly, 
these requirement^^ need further specif icatlon, a task that NIE should 
work on contlnuous^T 

1\ Functional specialization , « 

I have already mentioned this requirement^ and since it has become 
widely accepted and implemented, I wall not dwell on it here. - NIE 
should remain alerts hoyever, to emiergent needs, for new specialties 
and sub-specialties ^axid be j^repared to play a role in supporting-- 
their development. The most obvious mechanism is training programs 
and individual fellowships; another. wopld be "brokering" for people 
who ar^e just be^inning^to specialize in a particular role,. e.g., ' 
planners of innovative fiew schools, formatLve evalu^tors, local 
information^ of flcfers, regional extension agents'^ R&D inanagement 
copsultants, 6tcs At present this task is not ^belng performed , to 
my knowledge. * \ ' . - 
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2. Balance among: 

a. Functions (basic research, applied research, product develop- 
ment, systems development, dissemination, technical asslsta 

— training, evaluatlbn); 

b. Performers (social scientists, professional educators, prac- 
titioner s,_JlIK_si:;aff-^-atlier experts); 

c. Settings — universities, non-profits, profits; government 
labs, R&D centers, R&D teams; NIE (in-house %&5fearch) ; 

*d. Decision-makers and Influentlals — federal, state and local, 
education agencies; professionals and disciplines; employers 
and businessmen; minorities; 

e. Supply and demand (R&D resources .and the demand for these 
resources). <f , 

Anyone familiar with the history of educational R&D In the past 
fifty years Is aware of the severe, periodic imbalances that occurred 

, among these various functions, performers, settings, and participators 
in decision-making. Field service and testing were supreme for decades; 
then field-initiated research .had a few halcyon year^ under the Coop^r- 

- ative Research Act; th^ pfoduct" development was tremendously inflated; 
and tiow it appears that dissemination is being pushed to the fore. (In 
view of the new emphasis on dissemination, care should be taken to 
insure that we do not return to the days of field service in a vicious 
historical cycle.) Accompanying these extreme swings have been imbalanc 
among types l)f -^performers, settings, and "decisionTrmakers, with each 
sector that was_ formerly privileged becoming embittered as funds began 
to flow elsewhere. 3y kn awkward stretch of the imaginatloTi, this 
pattern m^ht be seen as b^ance over time ; .but do R&D needs really 
change this much from, one ^Viod of time .t,p. the next? 

I This point ralai^ the critical issues"^of the balance between the 
supply of educational knowledge, productsy , Systems, etc. and the 
demand for tfiese resources. Virtually all of our R&D effort has been 
devoted to increasing supply; but what do we know about the state of 
-demand? Do we assess it in any systematic fashion (a point to be * ^ 
fi,dlscussed later) or do we? await Congressional mandates and special- 
interest pressures to move us in different directions? And what is 
being done to stimulate demand and to make it more enlightened? Here 
is another domain of serious imbalance in the national R&D system. 

^ In sum, what is needed is a system which is more concurrently 

balanced with' regard to 'all of the components mentioned above. . 



3,.^ Integration (or interrelation) 



a. . Functions, 
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b. Settings, 

c. Performers, 

d. Decision^makers and influent ials , 

e. Supply and demand. 

-N^We know that it is foolish to assume that activities conducted in 
isolated settings by autonomous performers will conttibute to the com- 
mon goals of educational reform. And yet, NIE has not effectively 
promoted relationships between research and training, dievelopment and 
dissemination, dissemination and problem-solving, lab§, labs and SEA's, 
lat^s and universities, academics and practitioners, educational R&D 
leaders and special interest group leaders, or ev^n researchers in the 
same field. Here is a domain where statesmanlike 'leadership as well 
as imaginative administration is, I think, very badly nee<Jed. Purely 
contractual arrangements can be used' to further integration, e.g., 
requiring opportunities for meaningful apprenticeships in all research 
grants, ^requiring plans for linkages with SEA disseminators and other 
information centers in developmental contracts, requJfiring plans for 
coordination between programs and projects in the same problem area, 
etc. The mechanisms are numerous, but the main point is that integrat- 
ion should be focussed on as a distinct system requirement. In my ' 
Jtidgment, greater integration would not only improve the system as a 
whole, but also the functioning and outcomes of each component part. 

4. Continuity in: 

a. Policies, 

b. Tasks and substantive areas, 

c. Personnel, 

d. Organizations. ' ' . ^ 

It remains to be seen whether the dhronic discontinuities in 
educational R&D can be remedied without special attention to this 
system requirement. The history of R&D under USOE is rife with 
lessons. Almost every fiscal year brought a niew list of priorities, 
a new proportionate allocation among programs, j, a new threat of being 
terminated.' Turnover of personnel made it almost impossible for 
outsiders to continue a professional dialogue for more than two or 
three years or to maintain momentum for some projects which were 
'originally strongly endorsed. Occasionally, after a program had been 
announced and funded, by the time an investigator could get around to 
preparing a proposal the program no longer had funds or was in the 
throes of being ."phased out." Ironically, this is the picture that 
many have of NIE,, an agency which was set up to avoid' precisely these 
problems. 
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With regard to the R&D community at large, it is essential to 
continue structures and programs which are either too young to bear 
fruit or have already proven their worth. And even when some activi- 
ties must be terminated, every effort should be made to salvage the 
best parts by fostering new amalgams. These tactics are contrary to. 
the announced policy of creating a competitive struggle for existence 
among R&D performers. In all such struggles, the powerful and the 
cunning tend to triumph. 

5. Adaptability of: 

a. Policies, 

b. Functions, * / ^ 

c. Personnel (or criteria of selection). 

This. rtquirement might seem to be inconsistent with the need for 
continuity, and many of the discontinuities in federal educational R&D 
policy have been excused^ in ferms of the need for adaptation to new 
circumstances. But adaptability as a system requirement must not be 
confused with bending to pressure or embracing intellectual fads and 
ideologies. Adaptations can often be made or based upon on-going 
structures and present policy. For example, labs can be redu^ied in 
number and can be induced to give more attention to developing "systems 
rather than products, to link up with information dissemination centers, 
or to take marketing requirements into consideration during ^ the design 
phase; ERIC can become more active in raising awareness about educa- 
tions^ products; R&D centers can be helped to do a better training job; 
and so forth. The adaptability of existing structures can also be 
enhanced by providing funds for independent research, new conceptualiza- 
tions, planning of new departures, and conferences on needed research. 

The essential point, however, is that adaptation must be based 
on continuous , systematic inquiry into the diverse needs and problems 
of education rather than on the inspirations and ideologies of R&D 
decision-makers or the pressures of stakeholders. Such inquiry should 
be informed by (1) a s6t of explicit design requirements for the R&D 
system; (2) a continuous study of potential and actual resources, set- 
tings and perfomers of educational R&D (as begun with NIE' s new R&D 
Databook) ; ^ and (3) a continuous sensing of practitioners' needs, 
problems and reactions through systematic data-collection. (With ^ 
rfegard to the last point, a national "sensing netiiork" to identify tjie 
information needs of educators is now, being desiggled under the sponsor- 
ship of NIE' s dissemination division. This system could be easily 1 
adjusted to measure educational rieeds and monitor reactions to R&D 
products in general, thereby furnishing continuous, systematic data 
on<:^he needs and problems of practitioners throughout ^he nation. ^ 
The development and installation of thia system deserves the Council s 
fullest support, therefore.^) 
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In sum, a combination of system requirements with data on capacity , 
and data on needs should serve as the basis for all planning , short and 
long range. Much remains to be done to realize this triadic foundation 
of planning, however. Finally, aa suggested above, adaptations must be 
constrained by the requirements of balance, continuity and integration . 
These other system requirements set the boundaries of adaptability, in 
my opinion, and should not be substantially violated. 

6. ' Excellence or quality-control 

In the context of current pressures for dissemination, it is easy to 
forget that not all information or innovations are really worth dissem- 
inating. Obviously, if information or products are unreliable , mislead- 
ing, or unsuited to the situation, they can have repercussions which are 
altogether harmful. And even if not harmful, an accumulation of futile 
experiences with information or^ products might create an attitude of 
skepticism toward R&D of all kinds. Further, even when an innovation 
is workable and relatively effective, the opportunity cost of this parti- 
cular innovation rather than another might be undesirable. Although 
these points are obvious, it would seem that we are still wedded to the 
notion of "the more utilization the better." This assumption has 
prevented us from realizing that many of the so-called "barriers" and 
"resistances" to innovation and knowledge-utilization in American schools 
might actually be beneficial. 



The extent to which faddism dictates the adoption of innovations 
j)f doubtful merit has been demonstrated in a recent study of virtually 
all big-city secondary schools. This study reveals that while 46 per 
cent of these schools are relatively high in adoptions of innovations, 
most of the innovations adopted by half of these innovative schools are 
of relatively low quality (as judged by a national panel of secondary 
school experts). Clearly, quality remains as important an issue as 
quantity of adoptions. 

There are a host of discrete mechanisms for insuring that good work 
is done and disseminated; and yet, there seems to be continual debate 
over the quality of educational R&D. The source of this paradox might 
lie in the vagueness of evaluative criteria, the lack of consensus on 
procedures, and the failure of a major sponsor — NIE to develop any 
agency-wide mechanism for assessing quality. Perhaps what is vitally 
needed is a national task force or coitlmis&ion on the quality of educa- 
tional R&D . The mandates of such a commission might include: studying 
the ways in which quality is currently Assessed, the criteria best 
suited to different agencies, the qualifications of the assessors, the 
extent to which evaluation studies actually influence policy making, 
the methodology of evaluation rese^ch supported by NIE, the dangers of 
too stringent control, th« areas in which control is critical and tliose 
where it can be relaxed, and the dimensions of cost-benefit analysis. 
All functions (RDD&E) might be covered* or dif f erentssr task forced ctould 
concentrate on different functions. But whatever the mod^ o% operation, 
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it would seem that an external review of NIE's quality-control proced- 
ures would give coherence and focus to this vital design requirement. ' 

7. Goal setting 

A national R&D system must be able to formulate and gain consensus on 
a set of clear-cut objectives regarding output. Because these output 
goals were not clearly spelled out in the past, emphasis shifted from 
terminal to instrumental goals, or in simple language, from what the 
system was suppbsed to accomplish to how the .system was supposed to 
operate. Thus, the functions of RD&D (plus training) became the 
salient goals. Beyond these there was the single, overarching goal 
of "improving educational practice" or simply "impact." But what is 
meant by impact — awareness, enlightenment of practitioners, adoption, 
implementation, adaptation to local conditions, planning, improved 
learning? And what are the formal assets of impact that need to be 
kept in mind, such as scope, depth, duration, and effects? The ultimate 
goal of "impact on education" is, .too abstract and normative to communi- 
cate NIE's mission or to guide decision-making. 

\ Between this vague ultimate goal and the functions, of the^system 
itself there lies a vast no-man's land of inteTmediate ofejectives. 
These objectives need to be specified in operational language so that 
NIE personnel can use them for guidance and self-assessment, and s'o ^ 
that practitioners, legislators and the^R&D community will know what 
NIE is trying to accomplish. ^ Further, they must be prioritized so 
that resources are ncG^epleted and allocative decisions can be made 
with a minimum of debate. What are some of these intermediate output 
goals? Here are a few examples: \ 

\ . to gaiti participation of the ultimate consumer in C \ 
development, field testing and demtmstration; 

. to improve the discrimination of potential adopters 
of innovations ; . 

. to invent, install or nurture R&D systems in state 
and local settings; 

to improve, the problem-solving and needs-assessment 
capacity d'f ^.practitioners; y 

. to nourish the development of exemplary practices by 
practitioners; ^ 

^ 

to serve as a "leader" among federal agencies with 
activities related to educational R&D; 

.to encourage lateral communication among, schools 
engaged in similar innovations in the same region. 
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Not all goal-setting can or should be done by NIE, of course. 
Every agency that is engaged in the R&D process must be able to set 
goals that bear on the vital needs of ,ed\ication and that relate tt) 
the total R&D picture. Can NIE improve this process of pluralistic" 
goal-setting without undue pressure and control? And what mechanisms . 
are mo^ useful 'f9r gaining broad participation in NIE's own goal- 
setting? : ' \ 

To say that aii I^D system needs goals in plat;itudinous. To 
conceptualize? these goals in a manner which is both logical and 
precise, and to gain consensus on priorities, is a difficult task 
which, in my estimation^ has barely begun. ' . 

8* Recognition of environmental constraints and potential constituen- 
Q'ies (to gain realism and legitimacy^ ) 

The diversity, autonomy, and competition among the parts, as well as 
the political constraints and reward systems of the educational R&D ' 
community must be. taken into account in designing the components'df 
an R&b system. As the consultants' report: notes, educational R&D are 
secondary goals of many agencies and there is a relatively small cadre 
of full-time R&D performers, who are not necessarily the best-qualified 
Further, public education as a profession is low in prestige; inter- 
disciplinary work is rare; hucksterism and faddism are common; there> 
are ntunerods specialties and special interest groups with conflicting 
perspectives; practitioners resent the implication that they need to 
listen to "experts^' solve their "problems"; and so forth. Perhaps the 
best terms to summarize this environment are pluralism and conservatism 
(or preoccupation with non-R&D goals). My impressiop is that these 
features of the environment have not been adecjuately weighed in the 
formulation of R&D poli^^y^-atia programs. There has been a naive enthu-^ 
siasm for educational R&D which has not, and will never be, shared by 
the vast majority in several key groups: practicing^ educators, social 
scientists, activists, government officials, legislators^ arid la3nnen. 
This suggests two general approaches: (1) a cautious, realistic build- 
ing of individual programs with reference to exisjting constraints and 
strategic entry points; (2) a major attempt to gain grea^^r visibility 
and respect for educational R&D by means of a "showcase" pIxiBr am with ^ 
rWide participation and high impact. ^ — 

An example of i showcase program would be a national educational 
extension network. This approach appeals very much to practitioners 
because xt is oriented to their immediate needs and involved personal 
contact with someone^'who does not pose as an "expert." In addition, 
it raises .awareness ^f the diversity and riches of educational R&D 
resources. It has wide coverage, as a single extension agent can- 
reach approximately 300 clients per year. And it has been shown to ^ 
work very successfully in thtee states in a thoroiighly^evaluated 
USOE-Nl^ pilpt program. Not only educators generally but R&D 
personnel as^well would prof it^ from an extension system if it we^e 
designed , to serve as a massive on-going laboratory and a continuous 
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source of Information about the needs and reactions of educators to R&D 
pro]|ucts.. (And . eventually the system would redound to thi benefit of , 
basic inquiry as appreciation rose for R&D output.) Since^ .training 
materials and programs are available for preparing a national body of 
^extension agents and inf ormat^ion-retrieval personnel in sta^te and 
regional centers, the time that it would take install th^ program 
would be reduced to- a matter of mpnths. 

* It is my unAerstanding that NIE is currently trying to a^trengthen 
the capacity of state agencies to engage in linkage activities. But . 
t^±3 "bottom up" strategy, as important as it is, m^ight be too time 
consuming and produce'' a highly fragmented system which is Impdfesible 
to monitor. In view of the potential value * and timely relevance of 
a national extension system, a more aggressive "top-down" strategy 

might also be considered. ^ ^ * * 

^) 

In sum,, a showcase program of this kind ^(similar in visibiH^ty to ^ 
OEO's Headstart) might alleviate many of the political and professional 
problems that confront. the JJIE teday, and give it much greater connect- 
ion with our pluralistic educational environment.^ . 4 

Concluding remarks 

I do not* pretend to have covered within the brief compass of this li^pei; 
all the design requirements of an R&D system. I do believe that th^ 
ones I have mentioned are the most critical, however, and suf f ici^pt' 
as take-off , points for further elaboration. . . / 

Obviousl/, an important requirement which I havet oi^itted is she^t 
- money; and perhaps in ordpr to achieve the goals set forth in this p#er, 
"the level of funding for R&D would have to double or even triple. But 
' I have cast my eyes on the future and tried to imply that , the needs, of 
'an operating R&D system completely justify whatever resources are 
necessary to bring it about. If present f-unding restrictions continue, 
hbwyver, so that these systemic needs cannot be met, then I wbuld urge 
that we disabuse ourselves of the conceit that we are creating a 
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national educational R&D system." 
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i\B-<^^M^no^ov^t know by now, ten consultants to the Natip^ial Institute of 
Education have been asked to review present funtiing policies fol: educational 
^research alnd development, with particular references to >oli<fies that pay have ^, 
"an impact updn regional educational laboratories and rese^ch and^development 

centers. ^ - • 

to 

In order to be fully informed about the present state of eiiucational ^ 
research and ^developmerlt, the consultants ^re seeking information and ideas 
from a'wide^vajripty ofr sources. We held a face-to-face gathering on April 18 
in Chicago with leaders of laboratories and centers specifically, and we -plan , 
several on-site visits to learn more about laboratory and center activity first- 
h£^nd. ■ 

As Sam Sieber of our consultant group reviewed with the group of labora- 
tory and center people in Chicago, ^ we feel a need for certain iteims of infprma- 
tion from each organization individually. Therefore, we are sending you the 
enclosed ten questions for written reply. These witl allow you to give us the 
most current program, infoxrhation and funding picture (the precise and full ^ 
details of which may not be availab^ at NIE),. and will also allow you to ekpress 
candidly your individual opinions and preferences about research and develop- 
fnent funding policies'. We appreciated the -comments many of you made in 
Chicago, and we have revised these questions again to take your advice into 
accost.. 
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The intent of this request is not to gain a totally co;Tiprehensive picture 
of your organization, but to learn about certain aspects in which the group is 
most interested and to give you a chancy to highlight views you feel sho4ild be 
brought to our attention. We realize you could respond at any length to many 
of these items, with extensive and valuable infdrmation. Please do not 
nrii sunder stand our request for brief answers wherever possible; the consultants 
simply wiah to give 'each response personal attention, and the tlAie ^Uowed ue - 
by the NIR/Acting Director and the Council permits uer to review only limited, 
data frorr/Zeach of you. . « 

-^If Vou will return your responses to. all questions except number lO^to tny 
asaociatfe, Mr. P. Alistair Mac Kinnon, at the Washington Office of the New ' 
York Sfite Education Department, 1201 I6th Street, N. W« , Box. 100, Washington, ^ 
p. C. ^0036., we will be much obliged. ^ I enclose a postpaid jresponse envelope. 
For ypiir response to number 10, I enclose a second envelope sothat, as we Sig^ee4 
in Ct^hago, th^s infoTmation^^can pass directly to Sam Sieber in Confidence; A* 
prompt reply, no latet than Friday, May 9, 1975, will aid us in making use/of 
your /data. ' < , . , 



Please feel free to add^y thoughts to those requested, or to write to us 
latefr if you wish. ^ We appreciated your kind offers of help made hi Chicago, and 
we^/ thank you in advance for your assistance. 

/ .With renewed thank^s and best*wishes. 



Faithfully yours. 




^ald B. Nyquist 
On Behalf of the Coni^tant$ 
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^TUDY OF A LtERNA'-nVfi" FEDERAL RAiSEARCH AND DEVELOPN)iF.NT 

. • fundin<5^poli.gies ir^ ^:ducation-, , 

( REQUES'f F©R 'INFORMATION , 

\ ■ ■ .* * ■ f I » _ 

. April 24, 1975 

1. Description of programs - - 

a. Please describe your major activities --programs qr project8--ii 

educational ref,earch, development, evaluation, and dissemination. 
We would like to have the following information for ef ch majoy 
activity, displayed in t^e format shbwn on the attached sheet. 

" o Short title 

* • o Description of the activity (limited to a sentence pr two; tiilease 

' also classify tb^ activity^as to its mitjor';^ focus: resear^l 
development, 'evaluation, and dissemination) L 
o Total amount of fund$ that will have been inve^d in th* activity 

. by the tipie of presently scheduled' completion^ / 
o Source(s) of funds for each activity (name agency or o^h^r source(s) 
and indicate ^-pe of funding arrangement, tfuch as grint, contract, 
subcontract, |tc. ; include also method of funding, ^'sUch as ^ 
competitive aWard, Hon- competitive, '^tc. ) / 
o Number of current pr()fessional employees in the adtivity (FTEJ 
o Number of professional employees -(FTE) estimated to be working 

on the activity at time oSf presently scheduled completion 
o Number of employees vfko will be unable to be employed elsewhere 
s vn your organi^fcation upon termination of this actVvity (based on 
I presently known future funds) 

If your organization also engages in activities whicli are not directly 
related to educational ^res^earch, development, evaluation, and dis- 
^ semination (for example, training, media production, or other 
strictly service'activities), . please briefly desci/ibe these activities 
and their proportion of ydur organization's bUd^et. \ ' ^ 

s 

Budget summary ^ 

a. Using whatever 12-month period is your organization'^ fiscal yea^, 
please list in one chart the major activities ^upported'in the most 
Recently completed fiscal year, with the totil funds spent on each 

^ major activity in that year, and-the pe!f cent of the to,tal expense each 
^ activity represents. 

b. / Please show in a second unified display ea/ch of the major sources of 
funds to your organization during the fi^ame year .used in (a) a^bove, 
with amount of dollars from each, and t^e percent of total contributed 
by> each source. ' . / 



/ 
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■ . . • V ' 1 . ' ■;■ / ^. 

c- If. conditions during the. preseat fi^'cal year are substantially different.^ 
, from the conditions presented in the display under (a) and (b^ abov^, 
please ^briefly explain' the changes. ^ ' V'/ 

Needed funds Beginninja; with your next fiscal year; ^^hat NIE funds do yqu 
feel you realisticayy need in order to pursue the g^ls of your organizk- 
troo'^ * . . .... ' * * ^ , 

^ g'- . . * . • * ■ ' /$ \ 

Search for other funda Ar.e you pursuing any .olher^'source (s) of support 
beyond the NIE? I'f so^ould you tell us \yhich?' What influence has th'is - 
search for fCinds had on other activity in your , organization? Please be as 
specific ab ygu can on each. ~^ t - ' . - 

* ' - . ^ ■ y ' y . s « , . . 

Sources of. future fundj^ What is ♦your best estimate of your.fundir^ mix in 
your own fiscal ye'at 1976," from all sources ?V pi^se display a projected * 
set of sources, in a format like that Hn question 2(b). 

Desired continuations Whicl\ projects or major lines of work in your organ- • 
ization are most important tp continue beyond the curreVit y^ar, and why? 

Relatiotis with usgrs and outside, sources We feel it is important to under- 
stand how you coordinate your work with the perceptioais and policies of 
others stich as state and loc^al» educS-tion a^gencies, community groups, 
otheT laboratories and/or centers, scholars beyond your own staff, and 
other professionals. ^ ^ 

' a. How do groups intended to be eventual users of your prodycts or services 
influence the development, pf the progranris and projd|c%s undertaken by 
,yoxir organization? Has the degree of quality of thiS iijfluence changed 
in^ the past ie^r years^? Please indicate c'oncretely the impact of-such 
influence. ^ ' , ^ 

* . * ■ - 

b. Puring the past year, who was formally cons>ulted, other than your own 

staff or board/chie^ acad^emic administrator, on^matters of policy such 

\^ as beginning a new program or reviewin^^n operating program^ 

Please include" a list' of actual consultant^ used. 

. ' \ ^ ■ 

Help to others Beypnd your direct progxammatic activity ip research and 
product development intended for future benefit to schools ^rid education, 
how do y6u view your organizatioi) as a source of substantial* advice or 
technical assistance to people in other organizations oh problems related .* 
' to educational research, development, evaluation,, or dissemination?* 

a. Plea*: e discuss the role of your organization, at present, as a resource 
for people at the s^te and local education agency level,, or in schools, 
. ' aii<i indicate the level of effort devoted to this activity^ if s^ny. " 



^ 9; 



b. Please discuss the role o'f your organization as a resource for research 
and development perspnnel in other research organizations or univer-- 
sities, and indicate the level of effort devoted to this activity, if any. * 

Ratioilal^ f or institutional support WTiat special features^ of -your organization 
appear /o you to qualify it for ''institutional support'^ from the government ? 
^ecific form do you feel that suppprt should take, and why? 



What 



(NOTE: PuTyour. answer to this question on* separate sheets^, with the name 
of the responding organisation, and send it to Sam Sieber at bis home^ 
- address, as- detailed-in the covering letter accompanying this list of ques- 
tions. ) • ^ ' / 

I Relations with NIE ' What are thA major strengths and weaknesses ,in the 
' relati6nship between your orgainization and ^lE over the past two to three 

yeari?c (The more speci% you can make yojir response, the more.helpful 

it will be. ) . * * 



Please feel free to add any facts or views on these, or related, matters* 
which you fe«l the consultants sho\ild take into account^ 



'/Attachmente 



1. Format for taiple for program information, Cuestion 1. 
2*. Forniat for budget summary and fund-source summary. Question 2, and 
source of futuVe funds. Question 5. \ • , 

. 4 
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^alysis of Responses to Q. 10 'b5' Lab and Center Directors: 
"What aie the major strengths and weaknesses in the relation- 
ship between your organization and NIE over the past two or 
three years?" , . . ^ \ i . 



• This analysis is based on the following responses: 



ERIC 

hiiiinniirnrfTiaaaia 



Center^^directorb = 8 
Lab directors ^ =» 5 



1. , Althoiigh the question was word^ed to penriit the directors to 

mention bpth positive aYid negative aspects of their relation- 
ship with NIE,' the negative responses far outnumbered the 
posijClve oaes. Only 4 of the Centers mentioned a tolal of 7 
"strengths," as did only 3 of the Lab's. As one direrctor noted: ^ ^ 
"When this question was posed^to our key'staff, they Cgncluded ^ 
. / that they had great dl^ficylty in defining" any major strengths 
in the relationship between (tHe Lab) and NIE," All of the 
. "Centers and Labs metiticjjied "weaknesses," however ~ a total 
of 36 by the Centers and -34 by^the Labp. ^ ^ * . 

2. As shown in the above figures, .the Labs and Centers we^e equally ^ 
critical of NIE. Further, wi\h regard to only two negative, aspects 
were there somewhat more comments from one type of agency:, 4 Labs 

but only 2 Centers mentioned, personnel turnove'r as a problem; .and 

3 Lab« and only 1 Center mentioned a negative bias on the part of 

NIE 'towardk' these agencies. Ot.1f\erwise the comments of Lab and 

Center dfrkctors were virtually indistinguishable. Since there 

^ was little dif f erenJ;iation, our , tabulations below will not 

distinguish xJXese two kinds of agencies. • 

^ * • Dire^rtors 

3. The "strengths" mentioned by the directors were the folli)wing: . 

Competence, comntitmeht, help of NIE staff 



.Specific persons, e.g.', project officer, 

contracts officer, director or associate 
director . ' ' 

Specific group, e.^. pre-NIE staff, cer- 
tain programs ' . , 
In general * ' * 

' ' • , ^ o 

v 

Review, monitoring process ' 

In begihning of NIE- ^ ^ ' 

Recent^ improvement * 
• Numerous staff members' have visited 

^. " f g^^'^y • , - ^ * _ ■ -f 

^ ' ■ • 1 

Long-term contract ' - 

Recent evidence that NIE wishes to relate ^ 
to sea's ... 



9- ^ 
4^ 




Description of "strengths" in the relationship 

More than half of the directors ref^fcred to, the helpfulness v. 
and competence of NIE persbnnel >. It ie noteworthy, however, 
that they singled out particular persons rather than the staff 
as a whole* As one Lab direcfibr said, "The major strengths in 
(the Lab's) relationship with NIE ar^. gienerally attribu£a^>le to 
individual NIE staff members/' With an enthusiasm that was 
typical of several directors, one mentioned twc individuals in 
glowing terms:^ • 

,X, w^o has been our project officer f-or the last two o 
» years, 'has proved to be a highly inteWrigent, knowledge-* 
able, Qpen-minded and* decisive person • She is profea.- 
*sionally dedicated and is thoroughly honest about \ 
recognizing facts when facts are presented to her* 
- The second bright spot has been the presence and active 
leadership of ¥• We are extremely unhappy *that 'he is 
probably going to be leaving NIE within the next month or 
two* He is the one professionally trained /erson w^Haye 
encountered within NIE middle-management who ha^ bofh a 
_ substantive grasp of tTie field in which we work and a clear, 
purposeful vision of the kinds and quality of work, that need 
^ to be done ^o solve some of the highest priority problem^ in 
American education* 

This tendency to focus on individuals would seem to reflect an 
inability 'to oite virtues in NIE as an agency* Indeed, Several 
directors were'quite explicit ^^about this point* One mentioned 
the valuable "colleaguial relationships" with NiE staff which 
were being hampe;:ed by ^'bureaucratic rules ^" Another summed 
up the point as follows: 

NIE to date has been Ibss than "the sum of its parts* 
Qur relations with individuals within the Institute 
have been cordial and at a high professional level* 
^But by^ the time theise coma up out through the top they 
bear little resemblance to the interchange at the 
individual and unit level* *** For reasons "beyondT > 
their control" na one within the agency has been able 
to formulate and hold a position that is consistent 
with what fi'as been put fx)rtl\^as a "commitment" at an 
earlier time* - . ( ' - 

NeverthelMS,' the tendency of^ the directors to laud individual 
staff members reveals that the agency has a strong professional 
group to build upon^ That organizational problems might hamper 
the work of these individuals will be suggested later when we 
turn to the major "weaknesses" iri the relationship with NIE* 




The only other strength that was mentioned, by as many Tas 3 directors 
concerned' the revi'ew/monitorinR process . 'However, these coramentg 
were highly (qualified. ' One. director referxfed" only to. the beginning 
of NIE, and another said that J the review, process had improtved some- 
what in the'^'recent past. Two noted that, many staff members jDf NIE 
had visited their organizations over the years. * ' - 



The ''weaknesses" mentioned by Ithe directors were the following: 



Review, monitoring process ^ * 
Quantity 

Too frequent, burdensome, costly 



Quality 



Visitors* not competent tb judge 
Lack of consistent/priorities from NIE 
Poor standards - ^ 

Lack* of monitoring in recent years . 
Last ^iinute, .crisis-oriented reviews 
Need m compare products of labs-centers 
vs. ^others ^ ^ 



Discontinuity in policy, plans^ procedures, priorities ^ 
Commun i c a t i ons ® * ^ . 

Laek of involveme.nt of field, unresponsive to field 

Lack^of communication to field about decision-making ^ 
* process; top clique makes decisions in closed ^ 
manner; arbitrary decision^ imposed on managers 

Delays in reaching decisions, negotiating contracts , 
etc> ^ \* 

Poor internal communication in NIE, conflict ^mong 



N 

directors 



10 



,1 
2 
2 

t3 
1 



staff 



' Personnel turnover 

f 

Procurement 

" Dealing with individual projects, programs 
Competitive basis ' v ' .. 
No provision for start-ups> 
No funds' for f idld-initiated research ' 



13 



11 
'6 



4 
• 3 
2 

„ X 



Negative bias toward centers-labs, hostility 
iirfiompetgnce of staff (lack of training, experience) 

Clearance of research f orms 

* ' ■ — ^ • 

golitlcal ineptitude . > " 

Dissemination guidelines foo, restrictive 

Non-contract demands (^for. product descriptions, etc.^) 

o R&D model (linear, emphasizes research) • • * 

Failure to recognize distinction t)etweeri labg ang canters 

Exclusion of posts^condary education fro^ plans , 
especially regarding dissemination 
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Visits by consultants 

1. Regional educational laboratories • , . 

• ' a ■ ' 

Far West Laboratory for Educational Research and Development 
San Francisco I Xallfomla ' \ * , ^ * 

Southwest Educational development Laboratory 

'Austin, Texas * ^ 

Southyest Regional Laboratory for Educational Research and , . 
Development ' ' ^ ^ ' 

Los Alamltos, California' ^ * 

Central Midwestern Reg^nal Educational Laboratory, Inc. (CEMREL) 
St. Louis, Missouri 



2^ Research and development centeys^ 



Research and Development Center for Teachet Education 
University of X^x^s, Austin \^ . ^ 

Stanford, Center for Research and Development in Teaching 
Stanford University ^ > / 

Wisconsin Research and^dfeyelopment Center for Cognitive Learning 
University of WlsconLSith» Madison . 



3. ^ Other R&D performer* 

Center for Research and Development in Higher Education 
gnlvj^rslty of California, Berkeley 

Education Development Center, Inc. 
Newton, Massachusetts 

Stanford Research Institute 
Men lo Park, California f 

Educational Testing Service 
Princetoh, New Jersey 
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Personal Pteetlngs ^ ^ ' • . \ 

Americaif Association of Colleges of.Teather Education (Executive Director) 
American"* Associati6n of School Administrators (President-elect) ^ 
Amer\ican Educational Research Association (President, Ekecutive Director) 
Learning Research and Development .Center , University of - 

Pittsburgh (Director) 
National Education Association (group of staff) 

JIational Science Foundation, Education Directorate (grt)up of staff) 
B^searoh for Better Schools, Inc. (Director) ' • 

'U-.S. House of Representatives, Committee on Appropriations, ^ ^ 

Subcommittee on Departments of Labor and HEW (staff) 
U.S. Senate, Committee on Aj)propriatiofis, Subcommittee on Departments 
of Labor and HEW (staff) y " 

Teachers ColleRe, Columbia University (President) ^ 
Harvard Unives^sJ.ty Graduate School of Education (Dean and grou^ of faculty) 

Corresppndence^recei^d from individuals ^ . 

Stephen K. Bailey ' \ ' ^ 

American^ Council «n Education ^ ^ ^ a 

George Brain , * 

Washington State University " 

R. Louis Btight 

Western Institute for Science and Technology 
John Flanagan 

American Institutes for Research , - 

J. W. Getzels . 
University o/ Chicago 

Keith Gpldhafmer , 
Michigan State University ' ' 

Samuel Halperin * ^ 

Institute for (^ucational' Leadership . ^ 

Kenneth Hansen 

Nevada State Department of educat^ion ^ \ 

Lawrence Haskew, ; ^ 

University of Texas ' • % 



Richarc^ Herlig 

Kansas State Department of Education 
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Francis ianni 

Horace Manfi-^Lincoln Iifetitute, Teachers College Columbia ^University. 

^Francis ^Keppel • ^ 

AspjiBn Iilstitute for Humanistic Studies 

Kenneth Lindsay 

Utah State Board of Education * 

■ ■ * ■ * ■ 

•Sidney JMar land ' ^ ' 

* College Entrance Examination Board 

IWard S. Mason * ' ' . - . 7^ 

National institute of Education 

^ Frank Mattas , . , . . - 

Educational Resources Center - . 

Charles MojkowSki . 

Rhode Island Statfe Department of Educajtion 

,'Cary Potter - . . ' 

National Association of IndepejaJent Schools ' 

. " * '* . . * . . 

David Robinson > ' ^ 

Carnegie Corporation of New Wrk „ / *' 

Judith Segal 

National Institute of Education it , p 

Ralph Tyler' * , • 

Science Research Associates » , . u 



Theodore Waller 

Grolier Educational Corporation 

Sheldon Whitte 
Harvard University 
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